-- Logs begin at Tue 2022-08-30 00:36:41 UTC, end at Tue 2022-08-30 01:35:21 UTC. -- Aug 30 01:01:31.933474 dv-cinder-815145-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 30 01:01:31.942412 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 30 01:01:31.945364 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 30 01:01:31 2022] *** Aug 30 01:01:31.945594 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 30 01:01:31.945778 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 30 01:01:31.945938 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: nodename: dv-cinder-815145-12-iscsi Aug 30 01:01:31.946095 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: machine: x86_64 Aug 30 01:01:31.946248 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: clock source: unix Aug 30 01:01:31.946402 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: pcre jit disabled Aug 30 01:01:31.946562 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: detected number of CPU cores: 4 Aug 30 01:01:31.946709 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: current working directory: / Aug 30 01:01:31.946864 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: detected binary path: /usr/bin/uwsgi-core Aug 30 01:01:31.947025 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: your processes number limit is 47680 Aug 30 01:01:31.947181 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: your memory page size is 4096 bytes Aug 30 01:01:31.947337 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: detected max file descriptor number: 2048 Aug 30 01:01:31.947501 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: lock engine: pthread robust mutexes Aug 30 01:01:31.947806 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: thunder lock: enabled Aug 30 01:01:31.948201 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 30 01:01:31.948449 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 30 01:01:31.987967 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: Python main interpreter initialized at 0x560af81f59c0 Aug 30 01:01:31.987967 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: python threads support enabled Aug 30 01:01:31.987967 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: your server socket listen backlog is limited to 100 connections Aug 30 01:01:31.987967 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: your mercy for graceful operations on workers is 90 seconds Aug 30 01:01:31.988328 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: mapped 403077 bytes (393 KB) for 2 cores Aug 30 01:01:31.988328 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: *** Operational MODE: preforking *** Aug 30 01:01:31.988627 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: *** uWSGI is running in multiple interpreter mode *** Aug 30 01:01:31.988627 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: spawned uWSGI master process (pid: 81632) Aug 30 01:01:31.990933 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: spawned uWSGI worker 1 (pid: 81646, cores: 1) Aug 30 01:01:31.990933 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: spawned uWSGI worker 2 (pid: 81647, cores: 1) Aug 30 01:01:31.990933 dv-cinder-815145-12-iscsi devstack@keystone.service[81632]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 30 01:01:31.988831 dv-cinder-815145-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 30 01:01:36.323625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 01:01:36.323625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: from cryptography.utils import int_from_bytes Aug 30 01:01:36.324434 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 01:01:36.324434 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: from cryptography.utils import int_from_bytes Aug 30 01:01:36.324551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 01:01:36.324551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: from cryptography.utils import int_from_bytes Aug 30 01:01:36.324945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 01:01:36.324945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: from cryptography.utils import int_from_bytes Aug 30 01:01:36.375087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 01:01:36.375287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 01:01:36.375448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] command line args: [] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 01:01:36.375711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 01:01:36.375883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ================================================================================ {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 01:01:36.376719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] admin_token = **** {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.377044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] config_dir = [] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.377233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.377410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] config_source = [] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.377584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] debug = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.377779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.378952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.378952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.378952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.378952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.379178 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] list_limit = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.379178 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] log_config_append = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.379334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.379495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] log_dir = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.379667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] log_file = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.379880 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.380152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.380395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.380589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 01:01:36.380668 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.380791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 01:01:36.380866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.380989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.381074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] command line args: [] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 01:01:36.381183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.381283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 01:01:36.381387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ================================================================================ {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 01:01:36.381461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.381637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.381722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] admin_token = **** {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.381863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.381942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] config_dir = [] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.382054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.382126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.382250 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] config_source = [] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.382329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.382443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] debug = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.382522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.382636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.382742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.382819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.382970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.383053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.383224 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.383472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.383561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.383638 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] publish_errors = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.383759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] list_limit = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.383859 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.383946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] log_config_append = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.384057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.384148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.384245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] log_dir = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.384326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.384423 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] log_file = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.384512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.384659 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.384749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.384907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.384982 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] standard_threads = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.385133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.385218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.385368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.385449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.385541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.385621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.385712 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] use_journal = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.385796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.385867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] use_json = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.385956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.386032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] use_stderr = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.386197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] use_syslog = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.386270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.386420 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.386497 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.386654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.386747 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.386889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.386966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.387123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.387202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.387358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.387438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.387600 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.387683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.387831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.387907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.388095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.388328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.388466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.388542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] publish_errors = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.388610 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.388762 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.388848 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.388992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.389070 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.389171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.389247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.389418 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.389501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.389653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.389729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.389889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] standard_threads = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.389975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.390123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.390207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.390350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.390434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.390530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.390610 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.390706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] use_journal = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.390828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.390905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] use_json = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.391045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] use_stderr = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.391120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.391220 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] use_syslog = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.391295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.391451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:01:36.391528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.391707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.391783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.392631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.392714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.392877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.393055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.393138 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.393306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.393382 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.393527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.393614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.393757 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.393833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.393999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.394081 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.394227 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.394300 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.394450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.394526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.394674 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.394749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.394913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.394990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.395153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.395228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.395384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.395464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.395629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.395725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.395848 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.395926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.396099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.396177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.396328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.396406 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.396552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.396634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.396782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.396857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.397007 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.397089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.397231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.397366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.397451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.397548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.397687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.397811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.397949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.398066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.398141 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.398315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.398457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.398534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.398603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.398768 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.398878 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.398999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.399089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.399221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.399296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.399453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.399561 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.399682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.399833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.399906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.399981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.400150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.400227 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.400382 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.400468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.400607 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.400719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.400850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.400955 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.401080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.401193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.401336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.401458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.401556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.401642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.401759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.401843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.401988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.402074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.402211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.402291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.402441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.402529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.402669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.402745 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.402904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.402994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.403150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.403233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.403379 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.403504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.403616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.403693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.403844 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.403937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.404093 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.404212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.404311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.404385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.404573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.404667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.404789 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.404910 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.405020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.405096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.405255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.405334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.405485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.405564 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.405718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.405832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.405950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.406032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.406177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.406254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.406406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.406482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.406637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.406714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.406874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.406963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.407102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.407186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.407353 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.407429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.407628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.407696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.407859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.407935 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409421 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409543 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409543 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409543 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.409842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.410067 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.410195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.410304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.410390 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.410534 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.410617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.410777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.410861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.411009 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.411086 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.411257 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.411334 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.411511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.411712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.411843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.411920 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.412008 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] auth.external = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.412093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.412198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.412286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.412488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.412569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.412716 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.412793 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.412941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] auth.password = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.413018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.413167 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] auth.token = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.413245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.413402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.413479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.413633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.413710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.413874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.413995 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.414101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.414185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.414343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.414421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.414586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.414664 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.414811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.414899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.415061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.415304 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.415410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] credential.caching = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.415545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.415632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.415787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.415865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.416039 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.416121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.416276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.416366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.416510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.416588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.416748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.416837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.416980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.417066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.417220 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.417304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.417457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.417535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.417704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.417788 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.417942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.418019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.418464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.418549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.418720 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.418808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.418956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.419034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.419187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] auth.external = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.419263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.419420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.419507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] federation.caching = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.419649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.419731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.419888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.419966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.420150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.420246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] auth.password = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.420376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.420455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] auth.token = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.420617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.420701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.420847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.420933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.421093 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.421171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.421325 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.421401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.421573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.421659 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.421806 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.421884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.422040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.422116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.422269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.caching = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.422356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] credential.caching = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.422513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.422591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.422754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.422831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.422986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.423137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.423258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.423325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.423503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.423580 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.423740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.423820 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.423987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.424076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.424261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.424338 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.424500 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.424620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.424726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.424802 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.424952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.425030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.425178 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.425257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.425418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.425516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.425659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.425759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.425910 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.426015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.426135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.426252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] federation.caching = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.426376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.426489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.426640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.426719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.426881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.426958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.427113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.427190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.427340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.427419 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.427572 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.427649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.427796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.427874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.428046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.428129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.428286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.428385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.428520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.428626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.428764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.428876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.428990 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.429148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.caching = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.429230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.429401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.429488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.429652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.429730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.429894 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.429982 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.430134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.430220 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.430359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.430436 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.430593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.430670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.430817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.430902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.431051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.431198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.431275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.431461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.431538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.431699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.431778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.431934 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.432062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.432195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.432441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.432531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.432689 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.432766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.432916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.432992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.433142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.433230 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.433385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.433464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.433627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.433704 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.433858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.433937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.434091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.434168 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.434322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.434400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.434550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.434628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.434785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.434862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.435029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.435111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.435255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.435338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.435498 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.435575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.435737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.435814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.435967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.436106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.436223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.436300 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.436448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.436526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.436679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.436759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.436913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.436990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.437145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.437222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.437372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.437451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.437612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.437688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.437845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.437922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.438083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.438161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.438320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.438396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.438559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.439140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.439335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.439413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.439575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.439652 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.439832 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.439948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.440103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.440185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.440338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.440416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.440572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.440649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.440821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.440901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.441068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.441144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.441302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.441380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.441537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.441616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.441771 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.441847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.442001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.442077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.442233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.442311 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.442465 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.442543 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.442700 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.442777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.442947 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.443059 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.443196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.443302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.443446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.443523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.443683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.443762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.443913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.444007 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] resource.caching = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.444179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.444260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.444416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.444495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.444655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.444732 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.444887 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.444965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.445136 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.445215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.445383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.445460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.445626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.445704 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.445861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.445940 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.446098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.446175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.446325 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.446401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] role.caching = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.446567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] role.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.446644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.446793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.446871 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.447030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.447106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.447270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.447349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.447506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.447584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.447745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.447830 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.447987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.448126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.448245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.448323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.448485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.448574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.448718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.448795 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.448942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.449018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.449184 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.449337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.449401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.449459 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.449644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.449723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.449882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.449967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.450123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.450298 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.450384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.450532 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.450609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.450779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.450859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] resource.caching = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.451025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.451103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.451268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.451346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.451506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.451584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.451733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.451810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.451984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.452080 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.452247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.452327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.452481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.452558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.452709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.452787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.452936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.453016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.453162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.453240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] role.caching = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.453409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] role.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.453485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.453642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.453721 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.453885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.453962 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.454118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.454196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.454419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.455011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.455149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.455255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.455408 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.455495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] token.caching = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.455656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.455735 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.455902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.455981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.456176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.456268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.456414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.456492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.456663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.456741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.456906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.456992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.457148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.457227 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.457391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.457469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.457632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.457809 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.457889 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.458047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.458125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.458298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.458384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.458536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.458614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.458786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.458870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.459026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.459112 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.459262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.459342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.459520 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.459600 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.459772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.connection = **** {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.459849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.460023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.460116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.460262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.460340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.460493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.460570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.460724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.460809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.460953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.461030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.461184 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.461262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.461414 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.461499 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.461644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.461723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.461883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.461961 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.462114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.462200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.462349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] token.caching = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.462426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.462576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.462652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.462817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.462906 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.463047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.463125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.463284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.463361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.463507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.463592 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.463739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.463818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.463969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.464116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.464216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81647) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 01:01:36.464307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.464492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.464731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.464971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.465213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.465460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.465742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.465976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.466223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.466493 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.466757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.connection = **** {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.467002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.467251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.467490 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.467733 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.467970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.468232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.468469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.468708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.468953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.469191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.469437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.469673 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.470043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.470188 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.470435 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.470667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.470908 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.471145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:01:36.471316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81646) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 01:01:36.493712 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.493941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.494163 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.496138 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.496375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.496605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.502997 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.505801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.531669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.531894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.532485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.532722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.586448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.586922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.636237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.636458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.637564 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.637805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.794855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.795092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.795310 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.795479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.795642 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.795818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.795900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.795988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.796094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.796169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.796372 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.796617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.819383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.819633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.819864 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.819947 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.820111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.820215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.820331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.820415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.820548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.820630 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.820761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.820854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.820994 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.821211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.832568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.833199 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.846335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.846465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.846557 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.846634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.846738 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.846837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.846960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.847042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.847173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.847255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.847383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.847473 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.847593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81647) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.847675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81646) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:01:36.864766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.864971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 01:01:36.865078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.865165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 01:01:36.865311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.865510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.865593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 01:01:36.865712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 01:01:36.865797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.865987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.866069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.866249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.866330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.866505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.866783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.867070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.875975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.876697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.876697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.876868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.876948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.877022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.877201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.877523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.883754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.884532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.886636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.886917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.887197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.887471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.887554 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.887748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.887829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.888026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.888124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.888298 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.888380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.888566 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.888647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.888833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.888922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.889105 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.889212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.889366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.889448 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.889764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.890044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.890319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.912868 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.913967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.914319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.914598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.915433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.915720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.919347 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.919628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.919907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.920667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.920954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.921238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.926489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.926770 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.927686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.927981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.930745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.931034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.931337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.931628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.931900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.932055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.932353 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.932642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.932929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.933208 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.933487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.933762 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.933970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.934058 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.934261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.934547 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.950318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.950601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.952332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.952619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.954984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.957238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.958077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.958371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.960474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.960775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.962690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.964978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.965802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.968112 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.968895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.969193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.969472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.969750 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.970063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.970987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.972307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.972623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.972924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.973213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.973501 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.973783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.987060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.987345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.987612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.987876 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.989566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:36.989864 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.990135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.990418 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.994564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.996244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.997255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.997776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.998046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:36.998798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.000364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.000643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.000725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.000955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.001272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.001553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.001835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.003563 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.003861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.004283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.004574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.004875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.013057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.013339 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.013616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.013890 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.014163 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.014433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.014723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.015950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.016283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.016668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.017725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.018017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.018297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.018585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.030942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.034473 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.034625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.037407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.037511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.039183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.040330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.040673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.040952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.041254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.041771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.043636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.043931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.044264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.048010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.049590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.050988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.052447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.052533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.052743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.053026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.053301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.055455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.055748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.056067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.056349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.061133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.061425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.061709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.064657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.064954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.065240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.069444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81647) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.069730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.070008 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.070279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.070548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.070826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.071095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.071365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.071645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.071915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.072132 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81646) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:01:37.072278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.072424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.072538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.072711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.072793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.072988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.073069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81647) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.073261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.073533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.073816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.074104 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.074385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.074656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.074928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.075190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.075455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.075743 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81646) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:01:37.107593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560af81f59c0 pid: 81647 (default app) Aug 30 01:01:37.110161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560af81f59c0 pid: 81646 (default app) Aug 30 01:01:37.111262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa22da-00b0-429d-b101-ef9cbb7d724e None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:37.111548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa22da-00b0-429d-b101-ef9cbb7d724e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:37.111817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa22da-00b0-429d-b101-ef9cbb7d724e None None] PATH_INFO: `/v3` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:37.115977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1/1] 10.222.0.44 () {50 vars in 791 bytes} [Tue Aug 30 01:01:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:44.008236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d58dc8-d802-46eb-b603-1895528eaa9c None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:44.008769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d58dc8-d802-46eb-b603-1895528eaa9c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:44.008769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d58dc8-d802-46eb-b603-1895528eaa9c None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:44.013030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1/2] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:01:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:01:44.019005 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fd6bd-a437-4116-b577-ad2f29b437b0 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:44.019620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fd6bd-a437-4116-b577-ad2f29b437b0 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:44.019620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fd6bd-a437-4116-b577-ad2f29b437b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:44.091223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: INFO dbcounter [None req-ba2fd6bd-a437-4116-b577-ad2f29b437b0 None None] Registered counter for database keystone Aug 30 01:01:44.111605 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG oslo_db.sqlalchemy.engines [None req-ba2fd6bd-a437-4116-b577-ad2f29b437b0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81647) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 01:01:44.118294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writer thread running {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 01:01:44.306049 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG passlib.handlers.bcrypt [None req-ba2fd6bd-a437-4116-b577-ad2f29b437b0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81647) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 01:01:44.306356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG passlib.handlers.bcrypt [None req-ba2fd6bd-a437-4116-b577-ad2f29b437b0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81647) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 01:01:44.365463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ba2fd6bd-a437-4116-b577-ad2f29b437b0 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:44.367069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ba2fd6bd-a437-4116-b577-ad2f29b437b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:44.514453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 2/3] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:01:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 01:01:45.044259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d58dc8-d802-46eb-b603-1895528eaa9c None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:45.112779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: INFO dbcounter [None req-42d58dc8-d802-46eb-b603-1895528eaa9c None None] Registered counter for database keystone Aug 30 01:01:45.133572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG oslo_db.sqlalchemy.engines [None req-42d58dc8-d802-46eb-b603-1895528eaa9c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81646) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 01:01:45.139786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writer thread running {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 01:01:45.263857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d58dc8-d802-46eb-b603-1895528eaa9c None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:45.268948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018b0b3-da5e-44f1-aee2-91fd3c9175f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:45.269578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018b0b3-da5e-44f1-aee2-91fd3c9175f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:45.269860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018b0b3-da5e-44f1-aee2-91fd3c9175f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:45.270132 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018b0b3-da5e-44f1-aee2-91fd3c9175f7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:45.278406 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018b0b3-da5e-44f1-aee2-91fd3c9175f7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:45.332033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1018b0b3-da5e-44f1-aee2-91fd3c9175f7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 01:01:45.332313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018b0b3-da5e-44f1-aee2-91fd3c9175f7 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:45.342230 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 2/4] 10.222.0.44 () {58 vars in 1324 bytes} [Tue Aug 30 01:01:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:01:45.348730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fd6bd-a437-4116-b577-ad2f29b437b0 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:45.363524 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fd6bd-a437-4116-b577-ad2f29b437b0 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:45.368032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb7ce7-03bf-45d8-986e-3686df5e646d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:45.369127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb7ce7-03bf-45d8-986e-3686df5e646d admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:45.369400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb7ce7-03bf-45d8-986e-3686df5e646d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:45.369672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb7ce7-03bf-45d8-986e-3686df5e646d admin admin] PATH_INFO: `/v3/projects/236d122dbb6b4283a260bcccbdcd4a4a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:45.371209 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb7ce7-03bf-45d8-986e-3686df5e646d admin admin] RBAC: Authorizing `identity:get_project(project_id=236d122dbb6b4283a260bcccbdcd4a4a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:45.425515 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dafb7ce7-03bf-45d8-986e-3686df5e646d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 01:01:45.425893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb7ce7-03bf-45d8-986e-3686df5e646d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:45.430682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 3/5] 10.222.0.44 () {56 vars in 1207 bytes} [Tue Aug 30 01:01:45 2022] GET /identity/v3/projects/236d122dbb6b4283a260bcccbdcd4a4a => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:47.346387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c064748-b31b-47a9-a972-0db72acbcfed None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.346929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c064748-b31b-47a9-a972-0db72acbcfed None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.346929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c064748-b31b-47a9-a972-0db72acbcfed None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.351102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 3/6] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:01:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:47.356938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56e6d4-bd2b-48f2-ad65-2f5b47b608cf None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.357406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56e6d4-bd2b-48f2-ad65-2f5b47b608cf None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.357499 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56e6d4-bd2b-48f2-ad65-2f5b47b608cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.379085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-eb56e6d4-bd2b-48f2-ad65-2f5b47b608cf None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:47.380645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-eb56e6d4-bd2b-48f2-ad65-2f5b47b608cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:47.388589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 4/7] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:01:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 01:01:47.662434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c064748-b31b-47a9-a972-0db72acbcfed None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.675101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c064748-b31b-47a9-a972-0db72acbcfed None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.679345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1ec4c-ca63-461f-bfaf-2415cac27fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.679949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1ec4c-ca63-461f-bfaf-2415cac27fea admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.680245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1ec4c-ca63-461f-bfaf-2415cac27fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.680515 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1ec4c-ca63-461f-bfaf-2415cac27fea admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.688668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1ec4c-ca63-461f-bfaf-2415cac27fea admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.690983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1ec4c-ca63-461f-bfaf-2415cac27fea admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.700055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 4/8] 10.222.0.44 () {58 vars in 1324 bytes} [Tue Aug 30 01:01:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:01:47.706574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56e6d4-bd2b-48f2-ad65-2f5b47b608cf None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:47.715708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56e6d4-bd2b-48f2-ad65-2f5b47b608cf None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:47.720107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a57dab-1848-41c9-a92d-f742f16d9109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:47.720917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a57dab-1848-41c9-a92d-f742f16d9109 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:47.721194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a57dab-1848-41c9-a92d-f742f16d9109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:47.721459 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a57dab-1848-41c9-a92d-f742f16d9109 admin admin] PATH_INFO: `/v3/users/56560cd65ed74a0a9e8aff03655fb0e8` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:47.722761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a57dab-1848-41c9-a92d-f742f16d9109 admin admin] RBAC: Authorizing `identity:get_user(user_id=56560cd65ed74a0a9e8aff03655fb0e8)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:47.724629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a57dab-1848-41c9-a92d-f742f16d9109 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:47.726172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 5/9] 10.222.0.44 () {56 vars in 1198 bytes} [Tue Aug 30 01:01:47 2022] GET /identity/v3/users/56560cd65ed74a0a9e8aff03655fb0e8 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:01:49.768898 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ad395-7f1f-43a1-9b6a-eaa6e8eccaee None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:49.769449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ad395-7f1f-43a1-9b6a-eaa6e8eccaee None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:49.769538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ad395-7f1f-43a1-9b6a-eaa6e8eccaee None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:49.770913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 5/10] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:49.776680 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b08e2-ede4-4ac8-b068-1e1665125bfb None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:49.777215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b08e2-ede4-4ac8-b068-1e1665125bfb None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:49.777215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b08e2-ede4-4ac8-b068-1e1665125bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:49.797431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-847b08e2-ede4-4ac8-b068-1e1665125bfb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:49.799678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-847b08e2-ede4-4ac8-b068-1e1665125bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:49.809261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f919eda-ea15-4384-b295-7e59263dc978 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:49.809546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f919eda-ea15-4384-b295-7e59263dc978 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:49.809824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f919eda-ea15-4384-b295-7e59263dc978 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:49.811177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 6/11] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:01:49.816559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d6761-608d-4e26-8887-b057bf303cd3 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:49.816848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d6761-608d-4e26-8887-b057bf303cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:49.817126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d6761-608d-4e26-8887-b057bf303cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:49.859281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG passlib.handlers.bcrypt [None req-1a2d6761-608d-4e26-8887-b057bf303cd3 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81646) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 01:01:49.859664 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG passlib.handlers.bcrypt [None req-1a2d6761-608d-4e26-8887-b057bf303cd3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81646) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 01:01:49.882335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 6/12] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:01:49.909599 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-1a2d6761-608d-4e26-8887-b057bf303cd3 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:49.911180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1a2d6761-608d-4e26-8887-b057bf303cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:50.014673 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 7/13] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:01:50.155132 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b08e2-ede4-4ac8-b068-1e1665125bfb None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.181523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:50.266755 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b08e2-ede4-4ac8-b068-1e1665125bfb None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.286584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d6761-608d-4e26-8887-b057bf303cd3 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.302470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c7262-ddb2-49be-9cb6-2a19e459542e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.303128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c7262-ddb2-49be-9cb6-2a19e459542e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.303408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c7262-ddb2-49be-9cb6-2a19e459542e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.303685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c7262-ddb2-49be-9cb6-2a19e459542e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.315918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c7262-ddb2-49be-9cb6-2a19e459542e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-8d7c7262-ddb2-49be-9cb6-2a19e459542e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:01:50.324826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:01:50.325930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:01:50.326773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 7/14] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:01:50.331856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c7262-ddb2-49be-9cb6-2a19e459542e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.391241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d6761-608d-4e26-8887-b057bf303cd3 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.393815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:50.416630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:50.431720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a2a48-f5d2-402b-b3ff-673cfa71c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.432611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a2a48-f5d2-402b-b3ff-673cfa71c579 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.432888 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a2a48-f5d2-402b-b3ff-673cfa71c579 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.433155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a2a48-f5d2-402b-b3ff-673cfa71c579 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.436885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c7262-ddb2-49be-9cb6-2a19e459542e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.474776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8293123-2126-4a91-8fd5-f0a84e5f24fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.475386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8293123-2126-4a91-8fd5-f0a84e5f24fe None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.475662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8293123-2126-4a91-8fd5-f0a84e5f24fe None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.475930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8293123-2126-4a91-8fd5-f0a84e5f24fe None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.476628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8293123-2126-4a91-8fd5-f0a84e5f24fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:50.476927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8293123-2126-4a91-8fd5-f0a84e5f24fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.478318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8293123-2126-4a91-8fd5-f0a84e5f24fe None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.491380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 8/15] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:01:50.499820 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8293123-2126-4a91-8fd5-f0a84e5f24fe None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.507176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a2a48-f5d2-402b-b3ff-673cfa71c579 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.508744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a2a48-f5d2-402b-b3ff-673cfa71c579 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.617988 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8293123-2126-4a91-8fd5-f0a84e5f24fe None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.630508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 8/16] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 01:01:50.636992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a2a48-f5d2-402b-b3ff-673cfa71c579 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.640306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:50.667197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a924b6c-7476-494f-ade3-6fb61f6bacce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.667977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a924b6c-7476-494f-ade3-6fb61f6bacce None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.668321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a924b6c-7476-494f-ade3-6fb61f6bacce None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.668596 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a924b6c-7476-494f-ade3-6fb61f6bacce None admin] PATH_INFO: `/v3/users/56560cd65ed74a0a9e8aff03655fb0e8` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.670045 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a924b6c-7476-494f-ade3-6fb61f6bacce None admin] RBAC: Authorizing `identity:get_user(user_id=56560cd65ed74a0a9e8aff03655fb0e8)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.671638 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a924b6c-7476-494f-ade3-6fb61f6bacce None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.673277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 9/17] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/users/56560cd65ed74a0a9e8aff03655fb0e8 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:50.681840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a924b6c-7476-494f-ade3-6fb61f6bacce None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.706871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:50.771271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a2a48-f5d2-402b-b3ff-673cfa71c579 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.810796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a924b6c-7476-494f-ade3-6fb61f6bacce None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:50.813592 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb186c1-5dd7-40be-af4e-9ecdc3eb0cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.815315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb186c1-5dd7-40be-af4e-9ecdc3eb0cfd None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.815592 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb186c1-5dd7-40be-af4e-9ecdc3eb0cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.815903 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb186c1-5dd7-40be-af4e-9ecdc3eb0cfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.837618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb186c1-5dd7-40be-af4e-9ecdc3eb0cfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.839524 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb186c1-5dd7-40be-af4e-9ecdc3eb0cfd None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.850269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9fda2-5227-4e81-8b73-1069feedd506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:50.851034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9fda2-5227-4e81-8b73-1069feedd506 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:50.851344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9fda2-5227-4e81-8b73-1069feedd506 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:50.851696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9fda2-5227-4e81-8b73-1069feedd506 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:50.852951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9fda2-5227-4e81-8b73-1069feedd506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:50.854395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9fda2-5227-4e81-8b73-1069feedd506 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:50.855966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 10/18] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:01:50.864894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9fda2-5227-4e81-8b73-1069feedd506 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-feb186c1-5dd7-40be-af4e-9ecdc3eb0cfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:01:50.866265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:01:50.867525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:01:50.869099 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:01:50.869099 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:01:50.869099 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:01:50.869099 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 9/19] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:01:50.874550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb186c1-5dd7-40be-af4e-9ecdc3eb0cfd None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:50.944404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:50.958794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:50.984679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9fda2-5227-4e81-8b73-1069feedd506 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:51.003397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb186c1-5dd7-40be-af4e-9ecdc3eb0cfd None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:51.024198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550296a-db84-4da8-b9f8-1d1145d4fd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:51.024877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550296a-db84-4da8-b9f8-1d1145d4fd98 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:51.025196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550296a-db84-4da8-b9f8-1d1145d4fd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:51.025507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550296a-db84-4da8-b9f8-1d1145d4fd98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:51.026330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550296a-db84-4da8-b9f8-1d1145d4fd98 None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.domain.id=default, user.id=56560cd65ed74a0a9e8aff03655fb0e8) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:51.026670 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550296a-db84-4da8-b9f8-1d1145d4fd98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:51.028288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550296a-db84-4da8-b9f8-1d1145d4fd98 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:51.036435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 11/20] 10.222.0.44 () {56 vars in 1322 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/role_assignments?user.id=56560cd65ed74a0a9e8aff03655fb0e8&scope.domain.id=default&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 233 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:01:51.046226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e7e1e-3cd9-41e2-b3cd-d8b48ef3f5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:51.046857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e7e1e-3cd9-41e2-b3cd-d8b48ef3f5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:51.047134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e7e1e-3cd9-41e2-b3cd-d8b48ef3f5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:51.047404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e7e1e-3cd9-41e2-b3cd-d8b48ef3f5b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:51.048126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e7e1e-3cd9-41e2-b3cd-d8b48ef3f5b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:51.048500 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e7e1e-3cd9-41e2-b3cd-d8b48ef3f5b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:51.050167 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e7e1e-3cd9-41e2-b3cd-d8b48ef3f5b6 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:51.074589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 10/21] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:01:53.605073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196031ca-e990-4e00-befd-a58d74ac9daa None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.605581 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196031ca-e990-4e00-befd-a58d74ac9daa None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.605654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196031ca-e990-4e00-befd-a58d74ac9daa None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.606978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 12/22] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:53.612734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e329f-d448-4e6c-a3ee-83b296ed918b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.613654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e329f-d448-4e6c-a3ee-83b296ed918b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.614132 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e329f-d448-4e6c-a3ee-83b296ed918b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.625564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286a7d2-01a4-4193-b422-a1392fc624f1 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.625864 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286a7d2-01a4-4193-b422-a1392fc624f1 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.626114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286a7d2-01a4-4193-b422-a1392fc624f1 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.628772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 13/23] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:01:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:01:53.636787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba4b28-89e8-4a26-b321-bce6ac790910 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.637067 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba4b28-89e8-4a26-b321-bce6ac790910 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.637343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba4b28-89e8-4a26-b321-bce6ac790910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.642966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d37e329f-d448-4e6c-a3ee-83b296ed918b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:53.644546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d37e329f-d448-4e6c-a3ee-83b296ed918b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:53.672094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7eba4b28-89e8-4a26-b321-bce6ac790910 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:53.673430 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7eba4b28-89e8-4a26-b321-bce6ac790910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:53.813471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 11/24] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:01:53.843261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 14/25] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:01:53.935936 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d8b86-7e5a-484b-ad77-1e521455b9ea None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.936342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d8b86-7e5a-484b-ad77-1e521455b9ea None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.937033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d8b86-7e5a-484b-ad77-1e521455b9ea None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.938999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 12/26] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:01:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:53.945997 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0ec54-ed5e-4bc9-88e2-6e386b3681f6 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:53.946305 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0ec54-ed5e-4bc9-88e2-6e386b3681f6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:53.946561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0ec54-ed5e-4bc9-88e2-6e386b3681f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:53.979128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d3f0ec54-ed5e-4bc9-88e2-6e386b3681f6 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:53.980610 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d3f0ec54-ed5e-4bc9-88e2-6e386b3681f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:54.087899 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 15/27] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:01:54.249302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ab561-baa1-4a1f-bf15-bf938e3f3f93 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.249302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ab561-baa1-4a1f-bf15-bf938e3f3f93 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.249302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ab561-baa1-4a1f-bf15-bf938e3f3f93 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.249302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 13/28] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:54.255424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa963109-9df4-4158-862c-40c37d28e197 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.255769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa963109-9df4-4158-862c-40c37d28e197 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.255976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa963109-9df4-4158-862c-40c37d28e197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.269969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:54.286745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-aa963109-9df4-4158-862c-40c37d28e197 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:54.287970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-aa963109-9df4-4158-862c-40c37d28e197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:54.304999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ab561-baa1-4a1f-bf15-bf938e3f3f93 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:54.402897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:54.433365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 16/29] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:01:54.517540 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ab561-baa1-4a1f-bf15-bf938e3f3f93 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:54.567012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa963109-9df4-4158-862c-40c37d28e197 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:54.581531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755695c0-86cd-48b2-ae24-ceacb5d36031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.582186 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755695c0-86cd-48b2-ae24-ceacb5d36031 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.582460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755695c0-86cd-48b2-ae24-ceacb5d36031 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.582728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755695c0-86cd-48b2-ae24-ceacb5d36031 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.598153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755695c0-86cd-48b2-ae24-ceacb5d36031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-755695c0-86cd-48b2-ae24-ceacb5d36031 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:01:54.605237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:01:54.606791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 14/30] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:01:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:01:54.608256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755695c0-86cd-48b2-ae24-ceacb5d36031 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:54.834178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755695c0-86cd-48b2-ae24-ceacb5d36031 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:54.841900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:54.869393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa963109-9df4-4158-862c-40c37d28e197 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:54.876238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3390c-5b5d-4ec5-9dff-0e16ab478965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.876726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3390c-5b5d-4ec5-9dff-0e16ab478965 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.876907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3390c-5b5d-4ec5-9dff-0e16ab478965 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.877175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3390c-5b5d-4ec5-9dff-0e16ab478965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.888769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3390c-5b5d-4ec5-9dff-0e16ab478965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:54.890309 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3390c-5b5d-4ec5-9dff-0e16ab478965 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-2ed3390c-5b5d-4ec5-9dff-0e16ab478965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:01:54.908152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:01:54.909914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:01:54.911484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:01:54.911484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:01:54.911484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:01:54.911484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 15/31] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:01:54.911484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3390c-5b5d-4ec5-9dff-0e16ab478965 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:54.946306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f046d62-8144-40ec-b00a-8d4e1c23d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:54.947806 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f046d62-8144-40ec-b00a-8d4e1c23d481 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:54.948644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f046d62-8144-40ec-b00a-8d4e1c23d481 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:54.949374 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f046d62-8144-40ec-b00a-8d4e1c23d481 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:54.950877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f046d62-8144-40ec-b00a-8d4e1c23d481 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:54.953816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f046d62-8144-40ec-b00a-8d4e1c23d481 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:54.961959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:55.036785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3390c-5b5d-4ec5-9dff-0e16ab478965 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.074172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79033b4b-7cf4-4ce6-96c9-a424383749bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.074795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79033b4b-7cf4-4ce6-96c9-a424383749bd None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.075077 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79033b4b-7cf4-4ce6-96c9-a424383749bd None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.075350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79033b4b-7cf4-4ce6-96c9-a424383749bd None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.076038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79033b4b-7cf4-4ce6-96c9-a424383749bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:55.076350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79033b4b-7cf4-4ce6-96c9-a424383749bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.077799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79033b4b-7cf4-4ce6-96c9-a424383749bd None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.090971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 16/32] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:01:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:55.090971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79033b4b-7cf4-4ce6-96c9-a424383749bd None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.114879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:55.200030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 17/33] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Aug 30 01:01:54 2022] POST /identity/v3/roles => generated 219 bytes in 628 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:01:55.200030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f046d62-8144-40ec-b00a-8d4e1c23d481 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.200327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79033b4b-7cf4-4ce6-96c9-a424383749bd None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.243007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12074f8-6d05-49e8-ba22-b27e59700842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.243639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12074f8-6d05-49e8-ba22-b27e59700842 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.243920 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12074f8-6d05-49e8-ba22-b27e59700842 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.244229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12074f8-6d05-49e8-ba22-b27e59700842 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.245123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12074f8-6d05-49e8-ba22-b27e59700842 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.246796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12074f8-6d05-49e8-ba22-b27e59700842 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.336231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f046d62-8144-40ec-b00a-8d4e1c23d481 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.375759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730905a-0fda-4779-bed2-4947a11f2c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.377295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730905a-0fda-4779-bed2-4947a11f2c28 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.377578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730905a-0fda-4779-bed2-4947a11f2c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.377832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730905a-0fda-4779-bed2-4947a11f2c28 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.378522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730905a-0fda-4779-bed2-4947a11f2c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:55.378825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730905a-0fda-4779-bed2-4947a11f2c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.380726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730905a-0fda-4779-bed2-4947a11f2c28 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.385843 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 17/34] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Aug 30 01:01:55 2022] POST /identity/v3/roles => generated 225 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:55.387456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879eadbc-de43-44de-b69c-95c8bf7c8cd2 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.387740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879eadbc-de43-44de-b69c-95c8bf7c8cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.388059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879eadbc-de43-44de-b69c-95c8bf7c8cd2 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.389857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 18/35] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:55.390429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879eadbc-de43-44de-b69c-95c8bf7c8cd2 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.405164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 18/36] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:55.406617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8d0ec-7a7d-45cb-934a-b28f7c92c5fa None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.406853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8d0ec-7a7d-45cb-934a-b28f7c92c5fa None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.412915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8d0ec-7a7d-45cb-934a-b28f7c92c5fa None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.412915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 19/37] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:01:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:55.412915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b63dd-f297-422b-8a0e-da088bf56d55 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.412915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b63dd-f297-422b-8a0e-da088bf56d55 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.413296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b63dd-f297-422b-8a0e-da088bf56d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.443417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-fb9b63dd-f297-422b-8a0e-da088bf56d55 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:55.445084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-fb9b63dd-f297-422b-8a0e-da088bf56d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:55.508471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:55.508711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:55.533730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 20/38] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:01:55.540459 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b63dd-f297-422b-8a0e-da088bf56d55 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.547880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879eadbc-de43-44de-b69c-95c8bf7c8cd2 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.594629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9b1c2-5fa9-45a8-828a-14e2d6c9fcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.595283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9b1c2-5fa9-45a8-828a-14e2d6c9fcfc None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.595670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9b1c2-5fa9-45a8-828a-14e2d6c9fcfc None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.595941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9b1c2-5fa9-45a8-828a-14e2d6c9fcfc None admin] PATH_INFO: `/v3/users/56560cd65ed74a0a9e8aff03655fb0e8` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.597897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9b1c2-5fa9-45a8-828a-14e2d6c9fcfc None admin] RBAC: Authorizing `identity:get_user(user_id=56560cd65ed74a0a9e8aff03655fb0e8)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.599491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9b1c2-5fa9-45a8-828a-14e2d6c9fcfc None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.601672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 19/39] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:01:55 2022] GET /identity/v3/users/56560cd65ed74a0a9e8aff03655fb0e8 => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:55.602419 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6080ad-e57b-4d25-94f4-bf8550ee905b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.602703 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6080ad-e57b-4d25-94f4-bf8550ee905b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.602986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6080ad-e57b-4d25-94f4-bf8550ee905b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.627228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-cf6080ad-e57b-4d25-94f4-bf8550ee905b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:55.634347 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-cf6080ad-e57b-4d25-94f4-bf8550ee905b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:55.681891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b63dd-f297-422b-8a0e-da088bf56d55 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.727875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 20/40] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:01:55.727875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6080ad-e57b-4d25-94f4-bf8550ee905b None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.740640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25527a3-a758-4a65-8ae0-8d62f6c23832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.741778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25527a3-a758-4a65-8ae0-8d62f6c23832 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.742477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25527a3-a758-4a65-8ae0-8d62f6c23832 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.743268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25527a3-a758-4a65-8ae0-8d62f6c23832 None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.744610 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25527a3-a758-4a65-8ae0-8d62f6c23832 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.747564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25527a3-a758-4a65-8ae0-8d62f6c23832 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.805488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 21/41] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:01:55 2022] POST /identity/v3/projects => generated 301 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:55.818902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25527a3-a758-4a65-8ae0-8d62f6c23832 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.826716 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:55.860663 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:55.862230 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6080ad-e57b-4d25-94f4-bf8550ee905b None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:55.920369 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666376f3-d86f-432c-b501-ce339642daf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:55.921907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666376f3-d86f-432c-b501-ce339642daf8 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:55.922608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666376f3-d86f-432c-b501-ce339642daf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:55.923313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666376f3-d86f-432c-b501-ce339642daf8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:55.925251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666376f3-d86f-432c-b501-ce339642daf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:55.927989 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666376f3-d86f-432c-b501-ce339642daf8 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:55.930762 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 21/42] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:01:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:55.937413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666376f3-d86f-432c-b501-ce339642daf8 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:55.953179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25527a3-a758-4a65-8ae0-8d62f6c23832 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:56.084352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666376f3-d86f-432c-b501-ce339642daf8 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:56.107028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a5b94-a47e-43f7-8bfd-68759d64989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:56.107683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a5b94-a47e-43f7-8bfd-68759d64989b None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:56.107961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a5b94-a47e-43f7-8bfd-68759d64989b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:56.108262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a5b94-a47e-43f7-8bfd-68759d64989b None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:56.108946 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a5b94-a47e-43f7-8bfd-68759d64989b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:56.110348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a5b94-a47e-43f7-8bfd-68759d64989b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:56.122487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64131da4-96e7-462b-9283-5e62853148bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:56.123319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64131da4-96e7-462b-9283-5e62853148bc None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:56.123602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64131da4-96e7-462b-9283-5e62853148bc None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:56.123884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64131da4-96e7-462b-9283-5e62853148bc None admin] PATH_INFO: `/v3/domains/default/users/56560cd65ed74a0a9e8aff03655fb0e8/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:56.125889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64131da4-96e7-462b-9283-5e62853148bc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=56560cd65ed74a0a9e8aff03655fb0e8, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:56.127407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 22/43] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Aug 30 01:01:55 2022] POST /identity/v3/roles => generated 223 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:01:56.128131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64131da4-96e7-462b-9283-5e62853148bc None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:56.128234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a5b94-a47e-43f7-8bfd-68759d64989b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:56.147232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 22/44] 10.222.0.44 () {58 vars in 1361 bytes} [Tue Aug 30 01:01:55 2022] PUT /identity/v3/domains/default/users/56560cd65ed74a0a9e8aff03655fb0e8/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:01:56.257166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:56.266946 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a5b94-a47e-43f7-8bfd-68759d64989b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:56.314066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77204e-1a0c-499a-9497-c0bf60149eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:56.315549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77204e-1a0c-499a-9497-c0bf60149eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:56.315834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77204e-1a0c-499a-9497-c0bf60149eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:56.316144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77204e-1a0c-499a-9497-c0bf60149eb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:56.317385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77204e-1a0c-499a-9497-c0bf60149eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:56.318866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77204e-1a0c-499a-9497-c0bf60149eb6 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:56.320971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 23/45] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:01:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:56.326532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64131da4-96e7-462b-9283-5e62853148bc None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:56.336444 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:56.436396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64131da4-96e7-462b-9283-5e62853148bc None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:56.471287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddefd89-f89c-45be-b9e1-32e14586882e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:56.471922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddefd89-f89c-45be-b9e1-32e14586882e None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:56.472227 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddefd89-f89c-45be-b9e1-32e14586882e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:56.472500 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddefd89-f89c-45be-b9e1-32e14586882e None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:56.473189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddefd89-f89c-45be-b9e1-32e14586882e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:56.474800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddefd89-f89c-45be-b9e1-32e14586882e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:56.511772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 23/46] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:01:56 2022] POST /identity/v3/projects => generated 312 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:01:58.123831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514b795-a0c2-4029-b552-e4c5458456ec None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:58.124352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514b795-a0c2-4029-b552-e4c5458456ec None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:58.124539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514b795-a0c2-4029-b552-e4c5458456ec None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:58.126055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 24/47] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:01:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:58.133759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c845a-4286-4d28-adc1-e5c150720acc None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:58.134265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c845a-4286-4d28-adc1-e5c150720acc None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:58.134352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c845a-4286-4d28-adc1-e5c150720acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:58.163492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-850c845a-4286-4d28-adc1-e5c150720acc None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:58.165630 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-850c845a-4286-4d28-adc1-e5c150720acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:58.231371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:58.256329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 24/48] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:01:58.504756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e7fcf-6ab4-42e5-a3e2-070f1870bcfd None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:58.505052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e7fcf-6ab4-42e5-a3e2-070f1870bcfd None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:58.505336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e7fcf-6ab4-42e5-a3e2-070f1870bcfd None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:58.506804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 25/49] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:01:58.512560 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b581d88-7973-409d-8914-a6d257d55e0c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:58.512875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b581d88-7973-409d-8914-a6d257d55e0c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:58.513225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b581d88-7973-409d-8914-a6d257d55e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:58.534833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-5b581d88-7973-409d-8914-a6d257d55e0c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:01:58.536130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5b581d88-7973-409d-8914-a6d257d55e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:01:58.543766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e7fcf-6ab4-42e5-a3e2-070f1870bcfd None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:58.627280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 25/50] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:01:58.665485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e7fcf-6ab4-42e5-a3e2-070f1870bcfd None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:58.704110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47054344-8aab-43ac-87ee-ed03dfdd1b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:58.704833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47054344-8aab-43ac-87ee-ed03dfdd1b56 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:58.705549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47054344-8aab-43ac-87ee-ed03dfdd1b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:58.705549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47054344-8aab-43ac-87ee-ed03dfdd1b56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:58.717971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47054344-8aab-43ac-87ee-ed03dfdd1b56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-47054344-8aab-43ac-87ee-ed03dfdd1b56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:01:58.722991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:01:58.724028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:01:58.724792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 26/51] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:01:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:01:58.730387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b581d88-7973-409d-8914-a6d257d55e0c None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:58.928203 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47054344-8aab-43ac-87ee-ed03dfdd1b56 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:58.961173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:58.969164 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b581d88-7973-409d-8914-a6d257d55e0c None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:58.994819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:59.014528 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fdb00-61fb-4c6e-8f76-483a180f4180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:59.015167 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fdb00-61fb-4c6e-8f76-483a180f4180 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:59.015448 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fdb00-61fb-4c6e-8f76-483a180f4180 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:59.015725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fdb00-61fb-4c6e-8f76-483a180f4180 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:59.016483 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fdb00-61fb-4c6e-8f76-483a180f4180 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:59.016792 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fdb00-61fb-4c6e-8f76-483a180f4180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:59.018641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fdb00-61fb-4c6e-8f76-483a180f4180 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:59.028539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 26/52] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:01:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:01:59.035358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fdb00-61fb-4c6e-8f76-483a180f4180 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:59.076736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47054344-8aab-43ac-87ee-ed03dfdd1b56 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:59.122470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd58b49-2840-4e2f-bd18-79825c798df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:59.123134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd58b49-2840-4e2f-bd18-79825c798df7 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:59.123416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd58b49-2840-4e2f-bd18-79825c798df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:59.123687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd58b49-2840-4e2f-bd18-79825c798df7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:59.124960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd58b49-2840-4e2f-bd18-79825c798df7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:59.126492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd58b49-2840-4e2f-bd18-79825c798df7 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:59.128537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 27/53] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:01:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:01:59.135748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd58b49-2840-4e2f-bd18-79825c798df7 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:59.163103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fdb00-61fb-4c6e-8f76-483a180f4180 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:59.202528 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d07fa2-2c76-4454-8e8c-6ccdc8126f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:59.203141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d07fa2-2c76-4454-8e8c-6ccdc8126f6d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:59.203429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d07fa2-2c76-4454-8e8c-6ccdc8126f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:59.203701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d07fa2-2c76-4454-8e8c-6ccdc8126f6d None admin] PATH_INFO: `/v3/users/56560cd65ed74a0a9e8aff03655fb0e8` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:59.205746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d07fa2-2c76-4454-8e8c-6ccdc8126f6d None admin] RBAC: Authorizing `identity:get_user(user_id=56560cd65ed74a0a9e8aff03655fb0e8)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:59.207394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d07fa2-2c76-4454-8e8c-6ccdc8126f6d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:59.209264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 27/54] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:01:59 2022] GET /identity/v3/users/56560cd65ed74a0a9e8aff03655fb0e8 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:01:59.216224 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d07fa2-2c76-4454-8e8c-6ccdc8126f6d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:59.262924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:59.275116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd58b49-2840-4e2f-bd18-79825c798df7 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:59.275781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:01:59.315093 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8f089-11ce-4e4d-a9d3-414d8215af2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:59.315930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8f089-11ce-4e4d-a9d3-414d8215af2e None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:59.316241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8f089-11ce-4e4d-a9d3-414d8215af2e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:59.316518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8f089-11ce-4e4d-a9d3-414d8215af2e None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:59.317204 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8f089-11ce-4e4d-a9d3-414d8215af2e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:59.318731 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8f089-11ce-4e4d-a9d3-414d8215af2e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:59.337319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d07fa2-2c76-4454-8e8c-6ccdc8126f6d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:59.349204 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 28/55] 10.222.0.44 () {60 vars in 1140 bytes} [Tue Aug 30 01:01:59 2022] POST /identity/v3/projects => generated 298 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:01:59.380681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d364567-a68c-4d28-895f-51768c8bdc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:59.381360 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d364567-a68c-4d28-895f-51768c8bdc30 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:59.381648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d364567-a68c-4d28-895f-51768c8bdc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:59.381932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d364567-a68c-4d28-895f-51768c8bdc30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:59.383446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d364567-a68c-4d28-895f-51768c8bdc30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:59.385043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d364567-a68c-4d28-895f-51768c8bdc30 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:59.386835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 28/56] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:01:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:01:59.394885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8f089-11ce-4e4d-a9d3-414d8215af2e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:01:59.518069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8f089-11ce-4e4d-a9d3-414d8215af2e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:01:59.558661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aead33d-ed02-491d-90cd-66bc9c092713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:01:59.559525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aead33d-ed02-491d-90cd-66bc9c092713 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:01:59.559525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aead33d-ed02-491d-90cd-66bc9c092713 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:01:59.559811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aead33d-ed02-491d-90cd-66bc9c092713 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:01:59.560582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aead33d-ed02-491d-90cd-66bc9c092713 None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.domain.id=default, user.id=56560cd65ed74a0a9e8aff03655fb0e8) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:01:59.560893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aead33d-ed02-491d-90cd-66bc9c092713 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:01:59.562370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aead33d-ed02-491d-90cd-66bc9c092713 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:01:59.569549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 29/57] 10.222.0.44 () {56 vars in 1322 bytes} [Tue Aug 30 01:01:59 2022] GET /identity/v3/role_assignments?user.id=56560cd65ed74a0a9e8aff03655fb0e8&scope.domain.id=default&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 530 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:01.318222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471f1b2-76a1-4774-a61a-a01b00236c56 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:01.318761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471f1b2-76a1-4774-a61a-a01b00236c56 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:01.318761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471f1b2-76a1-4774-a61a-a01b00236c56 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:01.320150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 29/58] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:01.325323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81cfd6-9f8a-4a42-87cd-cebba292ad22 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:01.325755 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81cfd6-9f8a-4a42-87cd-cebba292ad22 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:01.325872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81cfd6-9f8a-4a42-87cd-cebba292ad22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:01.344476 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-8f81cfd6-9f8a-4a42-87cd-cebba292ad22 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:01.345709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8f81cfd6-9f8a-4a42-87cd-cebba292ad22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:01.347723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:01.414267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 30/59] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:01.691547 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471f1b2-76a1-4774-a61a-a01b00236c56 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:01.792443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471f1b2-76a1-4774-a61a-a01b00236c56 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:01.809096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:01.826546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303c955-658d-41ce-81d7-8a7b6619a99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:01.827173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303c955-658d-41ce-81d7-8a7b6619a99a None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:01.827449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303c955-658d-41ce-81d7-8a7b6619a99a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:01.827720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303c955-658d-41ce-81d7-8a7b6619a99a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:01.828846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303c955-658d-41ce-81d7-8a7b6619a99a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:01.830622 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303c955-658d-41ce-81d7-8a7b6619a99a None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:01.831987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 30/60] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:02:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:01.837671 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81cfd6-9f8a-4a42-87cd-cebba292ad22 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:01.936658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81cfd6-9f8a-4a42-87cd-cebba292ad22 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:01.970958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7ddb3-ab87-492b-abad-6ac19c2ffcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:01.971487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7ddb3-ab87-492b-abad-6ac19c2ffcb0 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:01.971766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7ddb3-ab87-492b-abad-6ac19c2ffcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:01.972062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7ddb3-ab87-492b-abad-6ac19c2ffcb0 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:01.972734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7ddb3-ab87-492b-abad-6ac19c2ffcb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:01.974174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7ddb3-ab87-492b-abad-6ac19c2ffcb0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:01.988346 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:02.002598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 31/61] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:02:01 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:05.157592 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69ee31-a263-48d6-8c74-b2849beb7462 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.158987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69ee31-a263-48d6-8c74-b2849beb7462 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.158987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69ee31-a263-48d6-8c74-b2849beb7462 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.160048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 31/62] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:05.165491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ed903-3bd5-4719-af7b-df033e88d89a None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.166328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ed903-3bd5-4719-af7b-df033e88d89a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.166818 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ed903-3bd5-4719-af7b-df033e88d89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.193277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6c5ed903-3bd5-4719-af7b-df033e88d89a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:05.194826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6c5ed903-3bd5-4719-af7b-df033e88d89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:05.285368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 32/63] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:05.311954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b95dc-74d3-432b-bd35-1e74fe01b8d9 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.312283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b95dc-74d3-432b-bd35-1e74fe01b8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.312541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b95dc-74d3-432b-bd35-1e74fe01b8d9 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.313914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 32/64] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:05.325827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baa32a-e8e4-4b34-a096-2cb5da77716c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.326119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baa32a-e8e4-4b34-a096-2cb5da77716c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.326449 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baa32a-e8e4-4b34-a096-2cb5da77716c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.353124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-76baa32a-e8e4-4b34-a096-2cb5da77716c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:05.354339 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-76baa32a-e8e4-4b34-a096-2cb5da77716c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:05.454417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 33/65] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:05.468398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93929e6c-572a-41f8-9092-797bbda1a9a0 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.468719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93929e6c-572a-41f8-9092-797bbda1a9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.468961 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93929e6c-572a-41f8-9092-797bbda1a9a0 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.470386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 33/66] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:05.476667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b54a43-f384-4e3e-a6af-3c729643061c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.476970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b54a43-f384-4e3e-a6af-3c729643061c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.477228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b54a43-f384-4e3e-a6af-3c729643061c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.505228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-f0b54a43-f384-4e3e-a6af-3c729643061c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:05.506454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f0b54a43-f384-4e3e-a6af-3c729643061c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:05.582772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:05.586969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93929e6c-572a-41f8-9092-797bbda1a9a0 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:05.606043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 34/67] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:05.621347 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2eafb3-5973-46ce-893e-5156bd68377e None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.621643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2eafb3-5973-46ce-893e-5156bd68377e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.621899 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2eafb3-5973-46ce-893e-5156bd68377e None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.623418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 35/68] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:02:05.633928 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcec010-9e84-4cbc-bde0-a7a087d43ca8 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.634229 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcec010-9e84-4cbc-bde0-a7a087d43ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.634512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcec010-9e84-4cbc-bde0-a7a087d43ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.684055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6fcec010-9e84-4cbc-bde0-a7a087d43ca8 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:05.687292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6fcec010-9e84-4cbc-bde0-a7a087d43ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:05.826062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 36/69] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:02:05.827282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcec010-9e84-4cbc-bde0-a7a087d43ca8 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:05.833746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93929e6c-572a-41f8-9092-797bbda1a9a0 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:05.882150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87a226-38b2-498a-bede-b89caa4d85c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:05.882848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87a226-38b2-498a-bede-b89caa4d85c3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.883130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87a226-38b2-498a-bede-b89caa4d85c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.883403 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87a226-38b2-498a-bede-b89caa4d85c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.897795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87a226-38b2-498a-bede-b89caa4d85c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-ea87a226-38b2-498a-bede-b89caa4d85c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:05.904026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:05.904929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:05.905686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 34/70] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:02:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:05.907012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f745e49-450c-4bb2-88c3-9e53665387d1 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.907012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f745e49-450c-4bb2-88c3-9e53665387d1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.907012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f745e49-450c-4bb2-88c3-9e53665387d1 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:05.909492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 35/71] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:05.913697 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f745e49-450c-4bb2-88c3-9e53665387d1 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:05.954719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcec010-9e84-4cbc-bde0-a7a087d43ca8 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:05.971564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:05.981018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:05.992374 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7f62b-f473-446a-acd0-c42b243ffd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:05.993021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7f62b-f473-446a-acd0-c42b243ffd36 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:05.993304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7f62b-f473-446a-acd0-c42b243ffd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:05.993579 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7f62b-f473-446a-acd0-c42b243ffd36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.004676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7f62b-f473-446a-acd0-c42b243ffd36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-55f7f62b-f473-446a-acd0-c42b243ffd36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:06.009701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:06.010573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 37/72] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:06.016074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b433a9-9fee-429b-828c-6931eba73f9a None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.016756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b433a9-9fee-429b-828c-6931eba73f9a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.016756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b433a9-9fee-429b-828c-6931eba73f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.031218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-59b433a9-9fee-429b-828c-6931eba73f9a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:06.032441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-59b433a9-9fee-429b-828c-6931eba73f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:06.062147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f745e49-450c-4bb2-88c3-9e53665387d1 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.105099 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab3a7f-a255-4c4c-8138-7d455f2b702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.105490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 38/73] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:06.106213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab3a7f-a255-4c4c-8138-7d455f2b702c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.106372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b433a9-9fee-429b-828c-6931eba73f9a None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.106915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab3a7f-a255-4c4c-8138-7d455f2b702c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.107357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab3a7f-a255-4c4c-8138-7d455f2b702c None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.108244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab3a7f-a255-4c4c-8138-7d455f2b702c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:06.108717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab3a7f-a255-4c4c-8138-7d455f2b702c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.110294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab3a7f-a255-4c4c-8138-7d455f2b702c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.120759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 36/74] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:06.121845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab3a7f-a255-4c4c-8138-7d455f2b702c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.216395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b433a9-9fee-429b-828c-6931eba73f9a None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.220181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:06.245448 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab3a7f-a255-4c4c-8138-7d455f2b702c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.254520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0593db-fd9d-421e-a08f-5fb3f35acfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.254721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:06.255157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0593db-fd9d-421e-a08f-5fb3f35acfd2 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.255436 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0593db-fd9d-421e-a08f-5fb3f35acfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.255707 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0593db-fd9d-421e-a08f-5fb3f35acfd2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.266766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0593db-fd9d-421e-a08f-5fb3f35acfd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-bc0593db-fd9d-421e-a08f-5fb3f35acfd2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:06.272211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:06.273068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:06.273816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 39/75] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:06.278823 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0593db-fd9d-421e-a08f-5fb3f35acfd2 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.287697 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf555e-c34c-4bd8-a3dc-738279b37fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.288463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf555e-c34c-4bd8-a3dc-738279b37fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.288757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf555e-c34c-4bd8-a3dc-738279b37fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.289021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf555e-c34c-4bd8-a3dc-738279b37fc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.289698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf555e-c34c-4bd8-a3dc-738279b37fc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:06.290006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf555e-c34c-4bd8-a3dc-738279b37fc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.291382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf555e-c34c-4bd8-a3dc-738279b37fc6 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.305313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 37/76] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:06.305313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf555e-c34c-4bd8-a3dc-738279b37fc6 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.385837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0593db-fd9d-421e-a08f-5fb3f35acfd2 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.412986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf555e-c34c-4bd8-a3dc-738279b37fc6 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.426482 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66a5b6-f1d7-4b11-a72d-f6516782451c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.427128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66a5b6-f1d7-4b11-a72d-f6516782451c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.427405 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66a5b6-f1d7-4b11-a72d-f6516782451c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.427678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66a5b6-f1d7-4b11-a72d-f6516782451c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.438109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66a5b6-f1d7-4b11-a72d-f6516782451c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-6b66a5b6-f1d7-4b11-a72d-f6516782451c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:06.443012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:06.443846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:06.444653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 40/77] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:06.449004 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66a5b6-f1d7-4b11-a72d-f6516782451c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.449401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0c1ad-c1b3-484d-80aa-d4f93a37c3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.450051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0c1ad-c1b3-484d-80aa-d4f93a37c3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.450332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0c1ad-c1b3-484d-80aa-d4f93a37c3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.450607 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0c1ad-c1b3-484d-80aa-d4f93a37c3b6 None admin] PATH_INFO: `/v3/users/56560cd65ed74a0a9e8aff03655fb0e8` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.451755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0c1ad-c1b3-484d-80aa-d4f93a37c3b6 None admin] RBAC: Authorizing `identity:get_user(user_id=56560cd65ed74a0a9e8aff03655fb0e8)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.453254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0c1ad-c1b3-484d-80aa-d4f93a37c3b6 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.454897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 38/78] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/users/56560cd65ed74a0a9e8aff03655fb0e8 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:06.459665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0c1ad-c1b3-484d-80aa-d4f93a37c3b6 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.476387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:06.491606 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:06.556584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66a5b6-f1d7-4b11-a72d-f6516782451c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.556767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0c1ad-c1b3-484d-80aa-d4f93a37c3b6 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.589421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2862611-30cd-4682-af9d-eb01161773e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.589628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c1508-f8a2-4414-a707-179f22303bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.590058 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2862611-30cd-4682-af9d-eb01161773e5 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.590150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c1508-f8a2-4414-a707-179f22303bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.590339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2862611-30cd-4682-af9d-eb01161773e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.590428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c1508-f8a2-4414-a707-179f22303bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.590612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2862611-30cd-4682-af9d-eb01161773e5 None admin] PATH_INFO: `/v3/users/1370863b9efe4da78c3d0201098c91b5` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.590697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c1508-f8a2-4414-a707-179f22303bd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.591320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c1508-f8a2-4414-a707-179f22303bd0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:06.591624 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c1508-f8a2-4414-a707-179f22303bd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.593054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c1508-f8a2-4414-a707-179f22303bd0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.602079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 41/79] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:06.602923 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c1508-f8a2-4414-a707-179f22303bd0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.606903 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2862611-30cd-4682-af9d-eb01161773e5 None admin] RBAC: Authorizing `identity:get_user(user_id=1370863b9efe4da78c3d0201098c91b5)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.608398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2862611-30cd-4682-af9d-eb01161773e5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.610081 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 39/80] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/users/1370863b9efe4da78c3d0201098c91b5 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:06.610846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2862611-30cd-4682-af9d-eb01161773e5 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.694431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:06.706759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:06.713327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c1508-f8a2-4414-a707-179f22303bd0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.716093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2862611-30cd-4682-af9d-eb01161773e5 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.747643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895aae97-879d-401f-915e-0a67f58bc3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.748305 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895aae97-879d-401f-915e-0a67f58bc3ee None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.748616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895aae97-879d-401f-915e-0a67f58bc3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.748862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895aae97-879d-401f-915e-0a67f58bc3ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.749939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895aae97-879d-401f-915e-0a67f58bc3ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.750255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e3ca7-6dad-49bf-aa7f-8832e346e7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.750859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e3ca7-6dad-49bf-aa7f-8832e346e7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.751134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e3ca7-6dad-49bf-aa7f-8832e346e7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.751403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895aae97-879d-401f-915e-0a67f58bc3ee None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.751495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e3ca7-6dad-49bf-aa7f-8832e346e7a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.752119 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e3ca7-6dad-49bf-aa7f-8832e346e7a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:06.752446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e3ca7-6dad-49bf-aa7f-8832e346e7a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.753166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 42/81] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:06.753764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895aae97-879d-401f-915e-0a67f58bc3ee None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.753855 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e3ca7-6dad-49bf-aa7f-8832e346e7a3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.762616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 40/82] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:06.763418 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e3ca7-6dad-49bf-aa7f-8832e346e7a3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.853616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895aae97-879d-401f-915e-0a67f58bc3ee None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.864867 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e3ca7-6dad-49bf-aa7f-8832e346e7a3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:06.889003 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af307f62-46b1-4596-8acf-5de58958f6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.889617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af307f62-46b1-4596-8acf-5de58958f6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.889896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af307f62-46b1-4596-8acf-5de58958f6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.890163 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af307f62-46b1-4596-8acf-5de58958f6c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.897528 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99c35e-0e76-4ddd-8bbb-bae04f1f1039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:06.898162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99c35e-0e76-4ddd-8bbb-bae04f1f1039 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:06.898444 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99c35e-0e76-4ddd-8bbb-bae04f1f1039 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:06.898719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99c35e-0e76-4ddd-8bbb-bae04f1f1039 None admin] PATH_INFO: `/v3/users/1370863b9efe4da78c3d0201098c91b5` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:06.899860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99c35e-0e76-4ddd-8bbb-bae04f1f1039 None admin] RBAC: Authorizing `identity:get_user(user_id=1370863b9efe4da78c3d0201098c91b5)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.901334 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99c35e-0e76-4ddd-8bbb-bae04f1f1039 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:06.902939 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 41/83] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/users/1370863b9efe4da78c3d0201098c91b5 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:06.903708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99c35e-0e76-4ddd-8bbb-bae04f1f1039 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:06.915866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:06.931629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:06.961988 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af307f62-46b1-4596-8acf-5de58958f6c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:06.963503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af307f62-46b1-4596-8acf-5de58958f6c7 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.008337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99c35e-0e76-4ddd-8bbb-bae04f1f1039 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.047232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bd07e-5549-446c-8517-731b396c9522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.047859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bd07e-5549-446c-8517-731b396c9522 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.048172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bd07e-5549-446c-8517-731b396c9522 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.048450 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bd07e-5549-446c-8517-731b396c9522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.065783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 43/84] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:07.066629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af307f62-46b1-4596-8acf-5de58958f6c7 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.118634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bd07e-5549-446c-8517-731b396c9522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.120029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bd07e-5549-446c-8517-731b396c9522 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.121758 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:07.144905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:07.167760 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af307f62-46b1-4596-8acf-5de58958f6c7 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.202559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1431bdb-8674-44d5-be97-c5a4116cbd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.203444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1431bdb-8674-44d5-be97-c5a4116cbd91 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.203726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1431bdb-8674-44d5-be97-c5a4116cbd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.204016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1431bdb-8674-44d5-be97-c5a4116cbd91 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.204683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1431bdb-8674-44d5-be97-c5a4116cbd91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.206882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1431bdb-8674-44d5-be97-c5a4116cbd91 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.231049 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 44/85] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:02:07 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:07.232518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1431bdb-8674-44d5-be97-c5a4116cbd91 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.233645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 42/86] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:07.234095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bd07e-5549-446c-8517-731b396c9522 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.354109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1431bdb-8674-44d5-be97-c5a4116cbd91 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.366213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:07.381510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bd07e-5549-446c-8517-731b396c9522 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.397555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc5c9b-0cc7-47a7-86ea-3d06da08039e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.398207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc5c9b-0cc7-47a7-86ea-3d06da08039e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.398493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc5c9b-0cc7-47a7-86ea-3d06da08039e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.398769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc5c9b-0cc7-47a7-86ea-3d06da08039e None admin] PATH_INFO: `/v3/users/1370863b9efe4da78c3d0201098c91b5` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.401532 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc5c9b-0cc7-47a7-86ea-3d06da08039e None admin] RBAC: Authorizing `identity:get_user(user_id=1370863b9efe4da78c3d0201098c91b5)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.403001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc5c9b-0cc7-47a7-86ea-3d06da08039e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.404222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 45/87] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:07 2022] GET /identity/v3/users/1370863b9efe4da78c3d0201098c91b5 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:07.404874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc5c9b-0cc7-47a7-86ea-3d06da08039e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.412176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:07.433082 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414293c6-3833-4f68-ac99-268b82dc8e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.433706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414293c6-3833-4f68-ac99-268b82dc8e13 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.433987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414293c6-3833-4f68-ac99-268b82dc8e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.434272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414293c6-3833-4f68-ac99-268b82dc8e13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.520425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414293c6-3833-4f68-ac99-268b82dc8e13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.522223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414293c6-3833-4f68-ac99-268b82dc8e13 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.546578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc5c9b-0cc7-47a7-86ea-3d06da08039e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.590199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19043b53-d54f-4735-9ae2-e44fc4a1a546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.590529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19043b53-d54f-4735-9ae2-e44fc4a1a546 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.590714 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19043b53-d54f-4735-9ae2-e44fc4a1a546 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.591136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19043b53-d54f-4735-9ae2-e44fc4a1a546 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.592682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19043b53-d54f-4735-9ae2-e44fc4a1a546 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.594434 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19043b53-d54f-4735-9ae2-e44fc4a1a546 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.597114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 46/88] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:07 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:07.598428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19043b53-d54f-4735-9ae2-e44fc4a1a546 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.646433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:07.651801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 43/89] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:07.653053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414293c6-3833-4f68-ac99-268b82dc8e13 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.682803 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:07.703278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19043b53-d54f-4735-9ae2-e44fc4a1a546 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.739899 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80459abc-d417-4822-a90c-f81e482a663e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.740569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80459abc-d417-4822-a90c-f81e482a663e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.740847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80459abc-d417-4822-a90c-f81e482a663e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.741120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80459abc-d417-4822-a90c-f81e482a663e None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.742285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80459abc-d417-4822-a90c-f81e482a663e None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.743796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80459abc-d417-4822-a90c-f81e482a663e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.745513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 47/90] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:07 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:07.746353 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80459abc-d417-4822-a90c-f81e482a663e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.770968 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414293c6-3833-4f68-ac99-268b82dc8e13 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.809366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4951c1d-b1ad-40b9-a5dd-de10e526cf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.810317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4951c1d-b1ad-40b9-a5dd-de10e526cf4e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.810775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4951c1d-b1ad-40b9-a5dd-de10e526cf4e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.811320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4951c1d-b1ad-40b9-a5dd-de10e526cf4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.858108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80459abc-d417-4822-a90c-f81e482a663e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:07.873437 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:07.892022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4951c1d-b1ad-40b9-a5dd-de10e526cf4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.893874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4951c1d-b1ad-40b9-a5dd-de10e526cf4e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.898276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbba57-a19c-4c4c-80f9-c7b9265c2351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:07.899171 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbba57-a19c-4c4c-80f9-c7b9265c2351 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:07.899556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbba57-a19c-4c4c-80f9-c7b9265c2351 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:07.899983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbba57-a19c-4c4c-80f9-c7b9265c2351 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:07.901081 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbba57-a19c-4c4c-80f9-c7b9265c2351 None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=8a6d5bbd6e874220b0d317fa4794df82, user.id=56560cd65ed74a0a9e8aff03655fb0e8) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:07.901499 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbba57-a19c-4c4c-80f9-c7b9265c2351 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:07.903117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbba57-a19c-4c4c-80f9-c7b9265c2351 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:07.911853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 48/91] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:07 2022] GET /identity/v3/role_assignments?user.id=56560cd65ed74a0a9e8aff03655fb0e8&scope.project.id=8a6d5bbd6e874220b0d317fa4794df82&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:07.913308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbba57-a19c-4c4c-80f9-c7b9265c2351 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:07.928408 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:08.039145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 44/92] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:08.040191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4951c1d-b1ad-40b9-a5dd-de10e526cf4e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:08.050258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbba57-a19c-4c4c-80f9-c7b9265c2351 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.104029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce06db-3145-4a8e-add4-a83a4225449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.108156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce06db-3145-4a8e-add4-a83a4225449d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.108321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce06db-3145-4a8e-add4-a83a4225449d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.108521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce06db-3145-4a8e-add4-a83a4225449d None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.111429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce06db-3145-4a8e-add4-a83a4225449d None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.112985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce06db-3145-4a8e-add4-a83a4225449d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.114870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 49/93] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:07 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:08.115691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce06db-3145-4a8e-add4-a83a4225449d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:08.204686 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:08.208188 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4951c1d-b1ad-40b9-a5dd-de10e526cf4e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.256406 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:08.270407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e80b2-745f-447d-bb55-bf4e0f0d4edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.271552 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e80b2-745f-447d-bb55-bf4e0f0d4edd None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.272025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e80b2-745f-447d-bb55-bf4e0f0d4edd None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.272468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e80b2-745f-447d-bb55-bf4e0f0d4edd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.273513 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e80b2-745f-447d-bb55-bf4e0f0d4edd None admin] RBAC: Adding query filter params (role.id=b07ea738cc204b2b8125bf4cd6d9c54a, scope.project.id=8a6d5bbd6e874220b0d317fa4794df82, user.id=1370863b9efe4da78c3d0201098c91b5) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:08.274031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e80b2-745f-447d-bb55-bf4e0f0d4edd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.275644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e80b2-745f-447d-bb55-bf4e0f0d4edd None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.284773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 45/94] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:08 2022] GET /identity/v3/role_assignments?user.id=1370863b9efe4da78c3d0201098c91b5&scope.project.id=8a6d5bbd6e874220b0d317fa4794df82&role.id=b07ea738cc204b2b8125bf4cd6d9c54a => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:08.285940 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e80b2-745f-447d-bb55-bf4e0f0d4edd None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:08.332104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce06db-3145-4a8e-add4-a83a4225449d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.434192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0301a08-6eea-4cba-8dd8-9249ab67406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.434861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0301a08-6eea-4cba-8dd8-9249ab67406a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.435150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0301a08-6eea-4cba-8dd8-9249ab67406a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.435429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0301a08-6eea-4cba-8dd8-9249ab67406a None admin] PATH_INFO: `/v3/projects/962aa8a1586e4f29a0e5a49b25c662ba` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.440990 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0301a08-6eea-4cba-8dd8-9249ab67406a None admin] RBAC: Authorizing `identity:get_project(project_id=962aa8a1586e4f29a0e5a49b25c662ba)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.442545 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0301a08-6eea-4cba-8dd8-9249ab67406a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.444826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 50/95] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:08 2022] GET /identity/v3/projects/962aa8a1586e4f29a0e5a49b25c662ba => generated 312 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:08.451121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0301a08-6eea-4cba-8dd8-9249ab67406a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:08.468692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e80b2-745f-447d-bb55-bf4e0f0d4edd None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.533909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d55e1-b791-402f-8d75-1f0109aabf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.534634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d55e1-b791-402f-8d75-1f0109aabf11 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.534923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d55e1-b791-402f-8d75-1f0109aabf11 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.535223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d55e1-b791-402f-8d75-1f0109aabf11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.535980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d55e1-b791-402f-8d75-1f0109aabf11 None admin] RBAC: Adding query filter params (role.id=7562948bd63d483cad68427819d434a9, scope.project.id=8a6d5bbd6e874220b0d317fa4794df82, user.id=1370863b9efe4da78c3d0201098c91b5) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:08.536323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d55e1-b791-402f-8d75-1f0109aabf11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.537846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d55e1-b791-402f-8d75-1f0109aabf11 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.548264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 46/96] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:08 2022] GET /identity/v3/role_assignments?user.id=1370863b9efe4da78c3d0201098c91b5&scope.project.id=8a6d5bbd6e874220b0d317fa4794df82&role.id=7562948bd63d483cad68427819d434a9 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:08.639394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0301a08-6eea-4cba-8dd8-9249ab67406a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:08.662793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:08.703390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173e631-d0c8-4be7-9e37-1899bfa70340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:08.704715 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173e631-d0c8-4be7-9e37-1899bfa70340 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:08.705289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173e631-d0c8-4be7-9e37-1899bfa70340 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:08.705782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173e631-d0c8-4be7-9e37-1899bfa70340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:08.706792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173e631-d0c8-4be7-9e37-1899bfa70340 None admin] RBAC: Adding query filter params (role.id=b07ea738cc204b2b8125bf4cd6d9c54a, scope.project.id=962aa8a1586e4f29a0e5a49b25c662ba, user.id=1370863b9efe4da78c3d0201098c91b5) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:08.707369 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173e631-d0c8-4be7-9e37-1899bfa70340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:08.709226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173e631-d0c8-4be7-9e37-1899bfa70340 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:08.722239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 51/97] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:08 2022] GET /identity/v3/role_assignments?user.id=1370863b9efe4da78c3d0201098c91b5&scope.project.id=962aa8a1586e4f29a0e5a49b25c662ba&role.id=b07ea738cc204b2b8125bf4cd6d9c54a => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:09.954474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c20df-cda2-4e0d-b2cb-d89514c140ec None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.955551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c20df-cda2-4e0d-b2cb-d89514c140ec None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.955551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c20df-cda2-4e0d-b2cb-d89514c140ec None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.957904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 47/98] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:09.962943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e660137-880e-489e-9d5b-a5487ad23864 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:09.963359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e660137-880e-489e-9d5b-a5487ad23864 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:09.963445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e660137-880e-489e-9d5b-a5487ad23864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:09.990307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-2e660137-880e-489e-9d5b-a5487ad23864 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:09.991510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2e660137-880e-489e-9d5b-a5487ad23864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:10.094211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 52/99] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:10.613395 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c754e3f-8113-4340-8901-b82dab8a07a8 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.613801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c754e3f-8113-4340-8901-b82dab8a07a8 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.613975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c754e3f-8113-4340-8901-b82dab8a07a8 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.615559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 48/100] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:10.621347 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53252b08-33cd-4a0d-baf5-fa4a3e5c5495 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.621632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53252b08-33cd-4a0d-baf5-fa4a3e5c5495 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.621906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53252b08-33cd-4a0d-baf5-fa4a3e5c5495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.638537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c754e3f-8113-4340-8901-b82dab8a07a8 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.656032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-53252b08-33cd-4a0d-baf5-fa4a3e5c5495 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:10.656032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-53252b08-33cd-4a0d-baf5-fa4a3e5c5495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:10.750442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:10.805705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 53/101] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:10.843169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c754e3f-8113-4340-8901-b82dab8a07a8 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:10.891411 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841958ee-886a-4b5d-86b5-1fd14d5c95c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:10.892472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841958ee-886a-4b5d-86b5-1fd14d5c95c1 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.893135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841958ee-886a-4b5d-86b5-1fd14d5c95c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.893606 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841958ee-886a-4b5d-86b5-1fd14d5c95c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.912957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841958ee-886a-4b5d-86b5-1fd14d5c95c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-841958ee-886a-4b5d-86b5-1fd14d5c95c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:10.918288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:10.919200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:10.920222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:10.920222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:10.920222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:10.920222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:10.920222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:10.920222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:10.920222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:10.920222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:10.920222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:10.920222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:10.920222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:10.920222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:10.922494 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 49/102] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:02:10.926480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb49ca-7a49-40bb-83aa-edbe8fb82faa None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.926766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb49ca-7a49-40bb-83aa-edbe8fb82faa None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.927039 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb49ca-7a49-40bb-83aa-edbe8fb82faa None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.928342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841958ee-886a-4b5d-86b5-1fd14d5c95c1 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:10.930345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 54/103] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:10.946224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147943-556c-49db-a71a-b5388178f19c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:10.946509 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147943-556c-49db-a71a-b5388178f19c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:10.946788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147943-556c-49db-a71a-b5388178f19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:10.981453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e8147943-556c-49db-a71a-b5388178f19c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:10.982653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e8147943-556c-49db-a71a-b5388178f19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:11.046582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:11.083711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841958ee-886a-4b5d-86b5-1fd14d5c95c1 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:11.112396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:11.134791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a900b54-491a-4bef-8c61-65e5e425fbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:11.135433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a900b54-491a-4bef-8c61-65e5e425fbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.135720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a900b54-491a-4bef-8c61-65e5e425fbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.136025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a900b54-491a-4bef-8c61-65e5e425fbe7 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.136701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a900b54-491a-4bef-8c61-65e5e425fbe7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:11.137008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a900b54-491a-4bef-8c61-65e5e425fbe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:11.138408 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a900b54-491a-4bef-8c61-65e5e425fbe7 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:11.141502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 55/104] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:11.157287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 50/105] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:11.163309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8147943-556c-49db-a71a-b5388178f19c None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.256027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a900b54-491a-4bef-8c61-65e5e425fbe7 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.313993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8147943-556c-49db-a71a-b5388178f19c None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:11.368771 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfecdfe-571d-4889-aadd-6ebc326a05b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:11.369433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfecdfe-571d-4889-aadd-6ebc326a05b7 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.369712 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfecdfe-571d-4889-aadd-6ebc326a05b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.369986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfecdfe-571d-4889-aadd-6ebc326a05b7 None admin] PATH_INFO: `/v3/users/c5c61e97018d4a15825261fec8237dad` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.397822 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfecdfe-571d-4889-aadd-6ebc326a05b7 None admin] RBAC: Authorizing `identity:get_user(user_id=c5c61e97018d4a15825261fec8237dad)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:11.399232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfecdfe-571d-4889-aadd-6ebc326a05b7 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:11.402148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 56/106] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:11 2022] GET /identity/v3/users/c5c61e97018d4a15825261fec8237dad => generated 289 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:11.406766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfecdfe-571d-4889-aadd-6ebc326a05b7 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.427890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a900b54-491a-4bef-8c61-65e5e425fbe7 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:11.475357 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:11.501272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e3336-7ec2-4e86-8788-9811c419c6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:11.502008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e3336-7ec2-4e86-8788-9811c419c6ae None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.502986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e3336-7ec2-4e86-8788-9811c419c6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.503269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e3336-7ec2-4e86-8788-9811c419c6ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.519773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e3336-7ec2-4e86-8788-9811c419c6ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-7b4e3336-7ec2-4e86-8788-9811c419c6ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:11.525084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:11.525923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:11.526708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 51/107] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:02:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:02:11.528103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5779c-868c-4461-b059-9fae65807f69 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.528103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5779c-868c-4461-b059-9fae65807f69 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.528103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5779c-868c-4461-b059-9fae65807f69 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.535022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 52/108] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:11.536024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2efbc-7a67-4fbf-ac6b-63de1d33b4eb None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.536360 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2efbc-7a67-4fbf-ac6b-63de1d33b4eb None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.536639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2efbc-7a67-4fbf-ac6b-63de1d33b4eb None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.537860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 53/109] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:11.538564 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2efbc-7a67-4fbf-ac6b-63de1d33b4eb None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.574732 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfecdfe-571d-4889-aadd-6ebc326a05b7 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:11.576660 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:11.613389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a93273-db41-4ff4-9714-7a40b281b170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:11.613665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a93273-db41-4ff4-9714-7a40b281b170 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.613904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a93273-db41-4ff4-9714-7a40b281b170 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.614174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a93273-db41-4ff4-9714-7a40b281b170 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.653079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2efbc-7a67-4fbf-ac6b-63de1d33b4eb None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:11.686727 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a93273-db41-4ff4-9714-7a40b281b170 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:11.688202 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c34cf-684b-4a82-8c7e-12e185f3000f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:11.688375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a93273-db41-4ff4-9714-7a40b281b170 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:11.688820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c34cf-684b-4a82-8c7e-12e185f3000f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.689101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c34cf-684b-4a82-8c7e-12e185f3000f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.689965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c34cf-684b-4a82-8c7e-12e185f3000f None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.690639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c34cf-684b-4a82-8c7e-12e185f3000f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:11.690942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c34cf-684b-4a82-8c7e-12e185f3000f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:11.692339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c34cf-684b-4a82-8c7e-12e185f3000f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:11.704174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 54/110] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:11.704174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2438ed4-6b4d-4325-89b4-45a0cdfbe969 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.704174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2438ed4-6b4d-4325-89b4-45a0cdfbe969 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.704174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2438ed4-6b4d-4325-89b4-45a0cdfbe969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.718480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:11.723337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d2438ed4-6b4d-4325-89b4-45a0cdfbe969 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:11.724588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d2438ed4-6b4d-4325-89b4-45a0cdfbe969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:11.796865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 57/111] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:11.796865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff464cd3-359a-4b49-a372-fd9e1959234d None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.796865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff464cd3-359a-4b49-a372-fd9e1959234d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.797203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 55/112] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:11.797271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff464cd3-359a-4b49-a372-fd9e1959234d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.797335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2438ed4-6b4d-4325-89b4-45a0cdfbe969 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.812716 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ff464cd3-359a-4b49-a372-fd9e1959234d None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:11.813904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ff464cd3-359a-4b49-a372-fd9e1959234d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:11.828819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:11.886275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 58/113] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:11.887550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f289e-d692-4871-ada7-e011fef0a219 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.887833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f289e-d692-4871-ada7-e011fef0a219 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.888134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f289e-d692-4871-ada7-e011fef0a219 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.889523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 59/114] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:11.890091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f289e-d692-4871-ada7-e011fef0a219 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:11.923745 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2438ed4-6b4d-4325-89b4-45a0cdfbe969 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:11.960817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:11.974663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ded072-6ea1-4357-bf12-a3ff8724bdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:11.975346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ded072-6ea1-4357-bf12-a3ff8724bdb8 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:11.975633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ded072-6ea1-4357-bf12-a3ff8724bdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:11.975907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ded072-6ea1-4357-bf12-a3ff8724bdb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:11.977547 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ded072-6ea1-4357-bf12-a3ff8724bdb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:11.980287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ded072-6ea1-4357-bf12-a3ff8724bdb8 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:11.982195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 56/115] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:02:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:11.982833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ded072-6ea1-4357-bf12-a3ff8724bdb8 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:12.029569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f289e-d692-4871-ada7-e011fef0a219 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.080914 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfffad-029f-46fc-9d85-87d99ca61dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.082023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfffad-029f-46fc-9d85-87d99ca61dbc None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.082577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfffad-029f-46fc-9d85-87d99ca61dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.083421 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfffad-029f-46fc-9d85-87d99ca61dbc None admin] PATH_INFO: `/v3/users/56560cd65ed74a0a9e8aff03655fb0e8` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.085945 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfffad-029f-46fc-9d85-87d99ca61dbc None admin] RBAC: Authorizing `identity:get_user(user_id=56560cd65ed74a0a9e8aff03655fb0e8)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:12.089145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfffad-029f-46fc-9d85-87d99ca61dbc None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:12.090999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 60/116] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:11 2022] GET /identity/v3/users/56560cd65ed74a0a9e8aff03655fb0e8 => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:12.092470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08954c33-5e09-4fa3-b349-71caf8b7e736 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.092767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08954c33-5e09-4fa3-b349-71caf8b7e736 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.093074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08954c33-5e09-4fa3-b349-71caf8b7e736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.117346 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-08954c33-5e09-4fa3-b349-71caf8b7e736 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:12.118587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-08954c33-5e09-4fa3-b349-71caf8b7e736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:12.123691 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:12.133543 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ded072-6ea1-4357-bf12-a3ff8724bdb8 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.174464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd249d-2825-4f03-b626-5e33dfdd4a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.175023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd249d-2825-4f03-b626-5e33dfdd4a3b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.175303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd249d-2825-4f03-b626-5e33dfdd4a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.175580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd249d-2825-4f03-b626-5e33dfdd4a3b None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.176771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd249d-2825-4f03-b626-5e33dfdd4a3b None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:12.178250 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd249d-2825-4f03-b626-5e33dfdd4a3b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:12.179957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 57/117] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:11 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:12.180752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd249d-2825-4f03-b626-5e33dfdd4a3b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:12.198765 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 61/118] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:12.199752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08954c33-5e09-4fa3-b349-71caf8b7e736 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:12.265674 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:12.295649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd249d-2825-4f03-b626-5e33dfdd4a3b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.316970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08954c33-5e09-4fa3-b349-71caf8b7e736 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.337122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf57ac3-b21e-4d9f-a16b-854165adc37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.337737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf57ac3-b21e-4d9f-a16b-854165adc37e None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.338013 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf57ac3-b21e-4d9f-a16b-854165adc37e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.338281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf57ac3-b21e-4d9f-a16b-854165adc37e None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.338930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf57ac3-b21e-4d9f-a16b-854165adc37e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:12.340418 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf57ac3-b21e-4d9f-a16b-854165adc37e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:12.359375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac5f22-47ef-48b8-a434-6a2b133d9865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.359999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac5f22-47ef-48b8-a434-6a2b133d9865 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.360385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac5f22-47ef-48b8-a434-6a2b133d9865 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.360666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac5f22-47ef-48b8-a434-6a2b133d9865 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.369500 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 58/119] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:02:12 2022] POST /identity/v3/projects => generated 302 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:12.373278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac5f22-47ef-48b8-a434-6a2b133d9865 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-31ac5f22-47ef-48b8-a434-6a2b133d9865 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:12.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:12.380634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:12.381428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 62/120] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:12.382076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf57ac3-b21e-4d9f-a16b-854165adc37e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:12.382148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac5f22-47ef-48b8-a434-6a2b133d9865 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:12.414351 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:12.523330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac5f22-47ef-48b8-a434-6a2b133d9865 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.525811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf57ac3-b21e-4d9f-a16b-854165adc37e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.540143 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:12.566999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c08063-1887-494d-ab9a-42730f51223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.567925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c08063-1887-494d-ab9a-42730f51223b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.568408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c08063-1887-494d-ab9a-42730f51223b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.568687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c08063-1887-494d-ab9a-42730f51223b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.571440 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9040c-68c2-4bd0-afee-4544ef935cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.571724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9040c-68c2-4bd0-afee-4544ef935cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.572029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9040c-68c2-4bd0-afee-4544ef935cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.572386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9040c-68c2-4bd0-afee-4544ef935cf8 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.573561 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9040c-68c2-4bd0-afee-4544ef935cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:12.575026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9040c-68c2-4bd0-afee-4544ef935cf8 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:12.576988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 59/121] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:12 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:12.577771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9040c-68c2-4bd0-afee-4544ef935cf8 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:12.584722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c08063-1887-494d-ab9a-42730f51223b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-b7c08063-1887-494d-ab9a-42730f51223b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:12.589621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:12.590461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:12.591217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 63/122] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:12.591829 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c08063-1887-494d-ab9a-42730f51223b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:12.671522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:12.690624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9040c-68c2-4bd0-afee-4544ef935cf8 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.712245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c08063-1887-494d-ab9a-42730f51223b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.728441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038beba9-897c-4939-b7f8-8bb8732a1137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.729061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038beba9-897c-4939-b7f8-8bb8732a1137 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.729339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038beba9-897c-4939-b7f8-8bb8732a1137 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.729611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038beba9-897c-4939-b7f8-8bb8732a1137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.730343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038beba9-897c-4939-b7f8-8bb8732a1137 None admin] RBAC: Adding query filter params (role.id=cd6b754c94e54de6b3bc5e17618f1e67, scope.project.id=8a6d5bbd6e874220b0d317fa4794df82, user.id=c5c61e97018d4a15825261fec8237dad) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:12.730653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038beba9-897c-4939-b7f8-8bb8732a1137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:12.732128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038beba9-897c-4939-b7f8-8bb8732a1137 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:12.738381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 60/123] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:12 2022] GET /identity/v3/role_assignments?user.id=c5c61e97018d4a15825261fec8237dad&scope.project.id=8a6d5bbd6e874220b0d317fa4794df82&role.id=cd6b754c94e54de6b3bc5e17618f1e67 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:12.738381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038beba9-897c-4939-b7f8-8bb8732a1137 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:12.764467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28988b-c14f-4f54-8ecd-d1da4f534e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.765194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28988b-c14f-4f54-8ecd-d1da4f534e24 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.765537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28988b-c14f-4f54-8ecd-d1da4f534e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.765878 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28988b-c14f-4f54-8ecd-d1da4f534e24 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.766652 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28988b-c14f-4f54-8ecd-d1da4f534e24 None admin] RBAC: Adding query filter params (name=member) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:12.767028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28988b-c14f-4f54-8ecd-d1da4f534e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:12.769603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28988b-c14f-4f54-8ecd-d1da4f534e24 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:12.779837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 64/124] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:12.785390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28988b-c14f-4f54-8ecd-d1da4f534e24 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:12.839051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:12.875070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038beba9-897c-4939-b7f8-8bb8732a1137 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.922602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcb9f5-6a50-4040-92fa-f7e70c1b3029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.923237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcb9f5-6a50-4040-92fa-f7e70c1b3029 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.923519 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcb9f5-6a50-4040-92fa-f7e70c1b3029 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.924191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcb9f5-6a50-4040-92fa-f7e70c1b3029 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.936616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28988b-c14f-4f54-8ecd-d1da4f534e24 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:12.938659 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcb9f5-6a50-4040-92fa-f7e70c1b3029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-41fcb9f5-6a50-4040-92fa-f7e70c1b3029 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:12.944040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:12.944922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:12.945699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 61/125] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:02:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:12.946316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcb9f5-6a50-4040-92fa-f7e70c1b3029 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:12.958383 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:12.994626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c7cdc-af89-4a7b-9889-3c47e9d259fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:12.995594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c7cdc-af89-4a7b-9889-3c47e9d259fb None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:12.996510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c7cdc-af89-4a7b-9889-3c47e9d259fb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:12.996941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c7cdc-af89-4a7b-9889-3c47e9d259fb None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82/users/56560cd65ed74a0a9e8aff03655fb0e8/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:12.999741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c7cdc-af89-4a7b-9889-3c47e9d259fb None admin] RBAC: Authorizing `identity:create_grant(project_id=8a6d5bbd6e874220b0d317fa4794df82, user_id=56560cd65ed74a0a9e8aff03655fb0e8, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.002449 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c7cdc-af89-4a7b-9889-3c47e9d259fb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.023381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 65/126] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:02:12 2022] PUT /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82/users/56560cd65ed74a0a9e8aff03655fb0e8/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:13.030979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c7cdc-af89-4a7b-9889-3c47e9d259fb None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.101609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcb9f5-6a50-4040-92fa-f7e70c1b3029 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.170487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e3d2f-bf53-4ca3-ac8e-895c9404c220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.171269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e3d2f-bf53-4ca3-ac8e-895c9404c220 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.171482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e3d2f-bf53-4ca3-ac8e-895c9404c220 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.171751 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e3d2f-bf53-4ca3-ac8e-895c9404c220 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.172503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e3d2f-bf53-4ca3-ac8e-895c9404c220 None admin] RBAC: Adding query filter params (name=member) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:13.172803 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e3d2f-bf53-4ca3-ac8e-895c9404c220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.176135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e3d2f-bf53-4ca3-ac8e-895c9404c220 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.195881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 62/127] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:13.197364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e3d2f-bf53-4ca3-ac8e-895c9404c220 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.235660 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:13.247838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c7cdc-af89-4a7b-9889-3c47e9d259fb None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.295741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e02fd-c4ae-43ca-ba09-114994d80b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.296365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e02fd-c4ae-43ca-ba09-114994d80b28 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.296646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e02fd-c4ae-43ca-ba09-114994d80b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.296921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e02fd-c4ae-43ca-ba09-114994d80b28 None admin] PATH_INFO: `/v3/users/1370863b9efe4da78c3d0201098c91b5` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.303897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e02fd-c4ae-43ca-ba09-114994d80b28 None admin] RBAC: Authorizing `identity:get_user(user_id=1370863b9efe4da78c3d0201098c91b5)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.303897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e02fd-c4ae-43ca-ba09-114994d80b28 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.303897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 66/128] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:13 2022] GET /identity/v3/users/1370863b9efe4da78c3d0201098c91b5 => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:13.303897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e02fd-c4ae-43ca-ba09-114994d80b28 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.323407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:13.375028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e3d2f-bf53-4ca3-ac8e-895c9404c220 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.453191 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e02fd-c4ae-43ca-ba09-114994d80b28 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.464725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9b3ea-e03f-427c-b1e3-ea799f3f23cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.465362 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9b3ea-e03f-427c-b1e3-ea799f3f23cc None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.465650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9b3ea-e03f-427c-b1e3-ea799f3f23cc None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.465928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9b3ea-e03f-427c-b1e3-ea799f3f23cc None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.466607 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9b3ea-e03f-427c-b1e3-ea799f3f23cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:13.466917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9b3ea-e03f-427c-b1e3-ea799f3f23cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.468351 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9b3ea-e03f-427c-b1e3-ea799f3f23cc None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.487721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 63/129] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:02:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:13.488623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9b3ea-e03f-427c-b1e3-ea799f3f23cc None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.492516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd0e08-a0b5-4beb-86d4-f0090a7425c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.493206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd0e08-a0b5-4beb-86d4-f0090a7425c2 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.493490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd0e08-a0b5-4beb-86d4-f0090a7425c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.493769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd0e08-a0b5-4beb-86d4-f0090a7425c2 None admin] PATH_INFO: `/v3/users/1370863b9efe4da78c3d0201098c91b5` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.495002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd0e08-a0b5-4beb-86d4-f0090a7425c2 None admin] RBAC: Authorizing `identity:get_user(user_id=1370863b9efe4da78c3d0201098c91b5)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.496637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd0e08-a0b5-4beb-86d4-f0090a7425c2 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.498302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 67/130] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:13 2022] GET /identity/v3/users/1370863b9efe4da78c3d0201098c91b5 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:13.499117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd0e08-a0b5-4beb-86d4-f0090a7425c2 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.534155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:13.609904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd0e08-a0b5-4beb-86d4-f0090a7425c2 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.664812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798c9fa-2fbb-446e-9b03-b884e6229bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.665549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798c9fa-2fbb-446e-9b03-b884e6229bda None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.665841 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798c9fa-2fbb-446e-9b03-b884e6229bda None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.666121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798c9fa-2fbb-446e-9b03-b884e6229bda None admin] PATH_INFO: `/v3/users/1370863b9efe4da78c3d0201098c91b5` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.667420 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798c9fa-2fbb-446e-9b03-b884e6229bda None admin] RBAC: Authorizing `identity:get_user(user_id=1370863b9efe4da78c3d0201098c91b5)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.668913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798c9fa-2fbb-446e-9b03-b884e6229bda None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.671154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 68/131] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:13 2022] GET /identity/v3/users/1370863b9efe4da78c3d0201098c91b5 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:13.671986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798c9fa-2fbb-446e-9b03-b884e6229bda None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.695364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9b3ea-e03f-427c-b1e3-ea799f3f23cc None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.731006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:13.739655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f3a71-ff2b-474f-bdf8-b66d662b67d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.741239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f3a71-ff2b-474f-bdf8-b66d662b67d7 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.741721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f3a71-ff2b-474f-bdf8-b66d662b67d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.742194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f3a71-ff2b-474f-bdf8-b66d662b67d7 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.743546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f3a71-ff2b-474f-bdf8-b66d662b67d7 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.745720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f3a71-ff2b-474f-bdf8-b66d662b67d7 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.747850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 64/132] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:13 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:13.748830 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f3a71-ff2b-474f-bdf8-b66d662b67d7 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.867729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f3a71-ff2b-474f-bdf8-b66d662b67d7 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:13.911334 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecb4bf-7688-44dd-a04d-99504fa67259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:13.912414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecb4bf-7688-44dd-a04d-99504fa67259 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:13.912894 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecb4bf-7688-44dd-a04d-99504fa67259 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:13.913362 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecb4bf-7688-44dd-a04d-99504fa67259 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:13.914902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecb4bf-7688-44dd-a04d-99504fa67259 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:13.916961 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecb4bf-7688-44dd-a04d-99504fa67259 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:13.919278 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 65/133] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:13 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:13.920566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecb4bf-7688-44dd-a04d-99504fa67259 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:13.961339 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:13.970170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798c9fa-2fbb-446e-9b03-b884e6229bda None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.003280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:14.047562 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecb4bf-7688-44dd-a04d-99504fa67259 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.077651 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dd26b-c60f-4c55-a6ef-1c6540d11eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.078382 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dd26b-c60f-4c55-a6ef-1c6540d11eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.078579 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dd26b-c60f-4c55-a6ef-1c6540d11eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.078845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dd26b-c60f-4c55-a6ef-1c6540d11eb6 None admin] PATH_INFO: `/v3/projects/962aa8a1586e4f29a0e5a49b25c662ba` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.084895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dd26b-c60f-4c55-a6ef-1c6540d11eb6 None admin] RBAC: Authorizing `identity:get_project(project_id=962aa8a1586e4f29a0e5a49b25c662ba)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.086416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dd26b-c60f-4c55-a6ef-1c6540d11eb6 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.088754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 69/134] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:13 2022] GET /identity/v3/projects/962aa8a1586e4f29a0e5a49b25c662ba => generated 312 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:14.089681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dd26b-c60f-4c55-a6ef-1c6540d11eb6 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.093144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e4d9d-151b-4c52-a696-b0809c7087a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.094166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e4d9d-151b-4c52-a696-b0809c7087a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.094660 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e4d9d-151b-4c52-a696-b0809c7087a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.095133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e4d9d-151b-4c52-a696-b0809c7087a2 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82/users/1370863b9efe4da78c3d0201098c91b5/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.098485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e4d9d-151b-4c52-a696-b0809c7087a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a6d5bbd6e874220b0d317fa4794df82, user_id=1370863b9efe4da78c3d0201098c91b5, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.101472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e4d9d-151b-4c52-a696-b0809c7087a2 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.135588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 66/135] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:02:13 2022] PUT /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82/users/1370863b9efe4da78c3d0201098c91b5/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:14.136757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e4d9d-151b-4c52-a696-b0809c7087a2 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:14.326392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dd26b-c60f-4c55-a6ef-1c6540d11eb6 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.377711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591ed08-8ff6-4417-a048-bb24599980e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.378427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591ed08-8ff6-4417-a048-bb24599980e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.378711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591ed08-8ff6-4417-a048-bb24599980e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.378991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591ed08-8ff6-4417-a048-bb24599980e6 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82/users/1370863b9efe4da78c3d0201098c91b5/roles/7562948bd63d483cad68427819d434a9` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.384372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591ed08-8ff6-4417-a048-bb24599980e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a6d5bbd6e874220b0d317fa4794df82, user_id=1370863b9efe4da78c3d0201098c91b5, role_id=7562948bd63d483cad68427819d434a9)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.386224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591ed08-8ff6-4417-a048-bb24599980e6 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.398198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e4d9d-151b-4c52-a696-b0809c7087a2 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:14.406510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 70/136] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:02:14 2022] PUT /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82/users/1370863b9efe4da78c3d0201098c91b5/roles/7562948bd63d483cad68427819d434a9 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:14.444399 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:14.451036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e59e9-404e-48fd-ac73-5c40676060d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:14.451814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e59e9-404e-48fd-ac73-5c40676060d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.452614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e59e9-404e-48fd-ac73-5c40676060d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.452904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e59e9-404e-48fd-ac73-5c40676060d3 None admin] PATH_INFO: `/v3/projects/962aa8a1586e4f29a0e5a49b25c662ba/users/1370863b9efe4da78c3d0201098c91b5/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.456095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e59e9-404e-48fd-ac73-5c40676060d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=962aa8a1586e4f29a0e5a49b25c662ba, user_id=1370863b9efe4da78c3d0201098c91b5, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:14.458552 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e59e9-404e-48fd-ac73-5c40676060d3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:14.474608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 67/137] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:02:14 2022] PUT /identity/v3/projects/962aa8a1586e4f29a0e5a49b25c662ba/users/1370863b9efe4da78c3d0201098c91b5/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:14.803827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8f4d2-f853-4737-975f-c56021a7d233 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.804181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8f4d2-f853-4737-975f-c56021a7d233 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.804423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8f4d2-f853-4737-975f-c56021a7d233 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.805854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 71/138] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:14.811662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e6723-4a44-4d35-b35a-6e0fec202c9d None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:14.812107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e6723-4a44-4d35-b35a-6e0fec202c9d None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:14.812292 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e6723-4a44-4d35-b35a-6e0fec202c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:14.840309 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-932e6723-4a44-4d35-b35a-6e0fec202c9d None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:14.841537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-932e6723-4a44-4d35-b35a-6e0fec202c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:14.934131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 68/139] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:02:15.201212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06261a-aeda-412e-94a6-b845906e60f6 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.202251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06261a-aeda-412e-94a6-b845906e60f6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.202748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06261a-aeda-412e-94a6-b845906e60f6 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.204392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 72/140] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:15.211050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cda311-ccd4-429c-bca9-9c1cd35de886 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.211375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cda311-ccd4-429c-bca9-9c1cd35de886 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.211611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cda311-ccd4-429c-bca9-9c1cd35de886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.231455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-14cda311-ccd4-429c-bca9-9c1cd35de886 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:15.232791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-14cda311-ccd4-429c-bca9-9c1cd35de886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:15.327319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 69/141] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:15.509573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06261a-aeda-412e-94a6-b845906e60f6 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:15.545373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:15.577733 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55718c76-ebf2-494c-b4aa-191951fa6d1f None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.578283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55718c76-ebf2-494c-b4aa-191951fa6d1f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.578283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55718c76-ebf2-494c-b4aa-191951fa6d1f None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.579974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 70/142] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:02:15.593645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d58b9f-3fa3-4142-9cd0-ac2316bb2229 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.593985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d58b9f-3fa3-4142-9cd0-ac2316bb2229 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.594216 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d58b9f-3fa3-4142-9cd0-ac2316bb2229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.628587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-70d58b9f-3fa3-4142-9cd0-ac2316bb2229 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:15.629844 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-70d58b9f-3fa3-4142-9cd0-ac2316bb2229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:15.664926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06261a-aeda-412e-94a6-b845906e60f6 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:15.724690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:15.765653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 71/143] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:02:15.772766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384178-9771-4328-87fb-e6e719b26a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:15.773472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384178-9771-4328-87fb-e6e719b26a72 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:15.773752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384178-9771-4328-87fb-e6e719b26a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:15.774025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384178-9771-4328-87fb-e6e719b26a72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:15.775278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa384178-9771-4328-87fb-e6e719b26a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:15.776784 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa384178-9771-4328-87fb-e6e719b26a72 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:15.778624 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 73/144] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:02:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:15.784869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d58b9f-3fa3-4142-9cd0-ac2316bb2229 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:15.856520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384178-9771-4328-87fb-e6e719b26a72 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:15.978303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d58b9f-3fa3-4142-9cd0-ac2316bb2229 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:15.992323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384178-9771-4328-87fb-e6e719b26a72 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:16.008392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:16.033452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65958dba-1119-4452-a71a-c1d492c39f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.033999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65958dba-1119-4452-a71a-c1d492c39f82 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.034440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65958dba-1119-4452-a71a-c1d492c39f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.034718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65958dba-1119-4452-a71a-c1d492c39f82 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.058673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65958dba-1119-4452-a71a-c1d492c39f82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-65958dba-1119-4452-a71a-c1d492c39f82 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:16.064810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:16.065994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:16.066837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 74/145] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:02:16.073343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65958dba-1119-4452-a71a-c1d492c39f82 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.089914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2157e9-d8bc-49aa-8aa3-f7bf75352857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.090547 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2157e9-d8bc-49aa-8aa3-f7bf75352857 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.090830 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2157e9-d8bc-49aa-8aa3-f7bf75352857 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.091165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2157e9-d8bc-49aa-8aa3-f7bf75352857 None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.091832 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2157e9-d8bc-49aa-8aa3-f7bf75352857 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:16.093315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2157e9-d8bc-49aa-8aa3-f7bf75352857 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:16.157713 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 72/146] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:02:15 2022] POST /identity/v3/users => generated 283 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:16.218433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2157e9-d8bc-49aa-8aa3-f7bf75352857 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.295943 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:16.298004 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65958dba-1119-4452-a71a-c1d492c39f82 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:16.370665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97f502-a5c7-47c6-8065-a7bf1df311b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.371291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97f502-a5c7-47c6-8065-a7bf1df311b5 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.371570 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97f502-a5c7-47c6-8065-a7bf1df311b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.371843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97f502-a5c7-47c6-8065-a7bf1df311b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.372557 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97f502-a5c7-47c6-8065-a7bf1df311b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:16.372860 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97f502-a5c7-47c6-8065-a7bf1df311b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:16.374266 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97f502-a5c7-47c6-8065-a7bf1df311b5 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:16.395110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 75/147] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:16.400845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97f502-a5c7-47c6-8065-a7bf1df311b5 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.438350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:16.451705 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2157e9-d8bc-49aa-8aa3-f7bf75352857 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:16.525820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a671b-a9b4-4158-9060-85c90cf9f38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.526474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a671b-a9b4-4158-9060-85c90cf9f38f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.526754 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a671b-a9b4-4158-9060-85c90cf9f38f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.527019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a671b-a9b4-4158-9060-85c90cf9f38f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.551939 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a671b-a9b4-4158-9060-85c90cf9f38f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-cc1a671b-a9b4-4158-9060-85c90cf9f38f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:16.557002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:16.557847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:16.558598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:16.558598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:16.558598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:16.558598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:16.558598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:16.558598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:16.558598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:16.558598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:16.558598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:16.558598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:16.558598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:16.558598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:16.562100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 73/148] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:02:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:02:16.566647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a671b-a9b4-4158-9060-85c90cf9f38f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.604149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97f502-a5c7-47c6-8065-a7bf1df311b5 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:16.670647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25eb7d-ebfc-422f-8dfe-0c020ffa837e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.671309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25eb7d-ebfc-422f-8dfe-0c020ffa837e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.671583 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25eb7d-ebfc-422f-8dfe-0c020ffa837e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.671854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25eb7d-ebfc-422f-8dfe-0c020ffa837e None admin] PATH_INFO: `/v3/users/c5c61e97018d4a15825261fec8237dad` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.677377 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25eb7d-ebfc-422f-8dfe-0c020ffa837e None admin] RBAC: Authorizing `identity:get_user(user_id=c5c61e97018d4a15825261fec8237dad)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:16.683772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25eb7d-ebfc-422f-8dfe-0c020ffa837e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:16.685673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 76/149] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:16 2022] GET /identity/v3/users/c5c61e97018d4a15825261fec8237dad => generated 289 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:16.710100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25eb7d-ebfc-422f-8dfe-0c020ffa837e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.781477 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:16.822511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a671b-a9b4-4158-9060-85c90cf9f38f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:16.851896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:16.866100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25eb7d-ebfc-422f-8dfe-0c020ffa837e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:16.911590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd2e67-cac9-4a57-92f5-330f4067419a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.912309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd2e67-cac9-4a57-92f5-330f4067419a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.912588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd2e67-cac9-4a57-92f5-330f4067419a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.912858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd2e67-cac9-4a57-92f5-330f4067419a None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.914032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd2e67-cac9-4a57-92f5-330f4067419a None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:16.915492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd2e67-cac9-4a57-92f5-330f4067419a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:16.917427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 77/150] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:16 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:16.923876 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd2e67-cac9-4a57-92f5-330f4067419a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:16.953613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f39a8f-22fe-4722-ba7b-82e0e185804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:16.953996 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39a8f-22fe-4722-ba7b-82e0e185804a None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:16.954283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39a8f-22fe-4722-ba7b-82e0e185804a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:16.954557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39a8f-22fe-4722-ba7b-82e0e185804a None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:16.955483 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f39a8f-22fe-4722-ba7b-82e0e185804a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:16.955761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f39a8f-22fe-4722-ba7b-82e0e185804a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:16.957229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f39a8f-22fe-4722-ba7b-82e0e185804a None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:16.986020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 74/151] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:16.991585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f39a8f-22fe-4722-ba7b-82e0e185804a None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.042498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd2e67-cac9-4a57-92f5-330f4067419a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.083516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c239f-15b3-4586-b1ef-6e2245e683a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.084555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c239f-15b3-4586-b1ef-6e2245e683a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.084987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c239f-15b3-4586-b1ef-6e2245e683a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.085561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c239f-15b3-4586-b1ef-6e2245e683a0 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82/users/c5c61e97018d4a15825261fec8237dad/roles/cd6b754c94e54de6b3bc5e17618f1e67` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.088398 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c239f-15b3-4586-b1ef-6e2245e683a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a6d5bbd6e874220b0d317fa4794df82, user_id=c5c61e97018d4a15825261fec8237dad, role_id=cd6b754c94e54de6b3bc5e17618f1e67)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.090031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c239f-15b3-4586-b1ef-6e2245e683a0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.107608 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 78/152] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:02:16 2022] PUT /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82/users/c5c61e97018d4a15825261fec8237dad/roles/cd6b754c94e54de6b3bc5e17618f1e67 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:17.194678 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f39a8f-22fe-4722-ba7b-82e0e185804a None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.202292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9bd06-6be5-46b1-bdb6-6e2ad487d852 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.202887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9bd06-6be5-46b1-bdb6-6e2ad487d852 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.203340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9bd06-6be5-46b1-bdb6-6e2ad487d852 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.205948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 79/153] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:17.212243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874afcb0-825e-49eb-adc8-a61e16a0302b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.212837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874afcb0-825e-49eb-adc8-a61e16a0302b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.213917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874afcb0-825e-49eb-adc8-a61e16a0302b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.252319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-874afcb0-825e-49eb-adc8-a61e16a0302b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:17.253542 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-874afcb0-825e-49eb-adc8-a61e16a0302b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:17.275181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:17.293870 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df645a-aa12-4253-b9e7-83a400a590c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.294536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df645a-aa12-4253-b9e7-83a400a590c0 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.294817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df645a-aa12-4253-b9e7-83a400a590c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.295350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df645a-aa12-4253-b9e7-83a400a590c0 None admin] PATH_INFO: `/v3/users/56560cd65ed74a0a9e8aff03655fb0e8` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.300956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df645a-aa12-4253-b9e7-83a400a590c0 None admin] RBAC: Authorizing `identity:get_user(user_id=56560cd65ed74a0a9e8aff03655fb0e8)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.302449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df645a-aa12-4253-b9e7-83a400a590c0 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.304180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 75/154] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:16 2022] GET /identity/v3/users/56560cd65ed74a0a9e8aff03655fb0e8 => generated 247 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:17.313301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df645a-aa12-4253-b9e7-83a400a590c0 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.370895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:17.405112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 80/155] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:17.406717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff9722-70bf-4a4d-bff9-b204505caf9a None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.406998 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff9722-70bf-4a4d-bff9-b204505caf9a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.407265 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff9722-70bf-4a4d-bff9-b204505caf9a None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.409618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 81/156] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:17.413629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef03cc-5e09-4daa-ab6a-0e1157756657 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.413921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef03cc-5e09-4daa-ab6a-0e1157756657 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.414199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef03cc-5e09-4daa-ab6a-0e1157756657 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.416389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 82/157] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:17.417654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f2fb5-8111-48b0-8f04-4dc391d98fff None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.418251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f2fb5-8111-48b0-8f04-4dc391d98fff None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.418573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f2fb5-8111-48b0-8f04-4dc391d98fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.446507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-c87f2fb5-8111-48b0-8f04-4dc391d98fff None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:17.447715 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c87f2fb5-8111-48b0-8f04-4dc391d98fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:17.521483 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df645a-aa12-4253-b9e7-83a400a590c0 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:17.570992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43841b4-c5c5-4048-9d51-d54de2a86ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:17.571625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43841b4-c5c5-4048-9d51-d54de2a86ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.571898 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43841b4-c5c5-4048-9d51-d54de2a86ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.572213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43841b4-c5c5-4048-9d51-d54de2a86ef2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.585179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 83/158] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:17.586844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5862f1-70b2-4cb6-a64b-0afb78292b53 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:17.587417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5862f1-70b2-4cb6-a64b-0afb78292b53 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:17.587941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5862f1-70b2-4cb6-a64b-0afb78292b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:17.613935 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-bd5862f1-70b2-4cb6-a64b-0afb78292b53 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:17.615514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-bd5862f1-70b2-4cb6-a64b-0afb78292b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:17.657004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43841b4-c5c5-4048-9d51-d54de2a86ef2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:17.659170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43841b4-c5c5-4048-9d51-d54de2a86ef2 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:17.698870 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:17.763586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 84/159] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:17.838056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 76/160] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 01:02:17.843764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5862f1-70b2-4cb6-a64b-0afb78292b53 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.853201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:17.941187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43841b4-c5c5-4048-9d51-d54de2a86ef2 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:17.973759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5862f1-70b2-4cb6-a64b-0afb78292b53 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:18.024218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bfd27-11cb-4cb6-b337-913354e31185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:18.025227 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bfd27-11cb-4cb6-b337-913354e31185 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:18.026296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bfd27-11cb-4cb6-b337-913354e31185 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:18.027632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bfd27-11cb-4cb6-b337-913354e31185 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:18.029672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55bfd27-11cb-4cb6-b337-913354e31185 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:18.032603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55bfd27-11cb-4cb6-b337-913354e31185 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:18.035043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 85/161] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:17 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:18.036224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bfd27-11cb-4cb6-b337-913354e31185 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:18.092617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43841b4-c5c5-4048-9d51-d54de2a86ef2 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:18.117853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:18.130747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bd9e4-8746-4a2c-b14b-51e26d7421a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:18.131412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bd9e4-8746-4a2c-b14b-51e26d7421a0 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:18.131687 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bd9e4-8746-4a2c-b14b-51e26d7421a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:18.131958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bd9e4-8746-4a2c-b14b-51e26d7421a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:18.145292 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5bd9e4-8746-4a2c-b14b-51e26d7421a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-ed5bd9e4-8746-4a2c-b14b-51e26d7421a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:18.150400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:18.151317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:18.152512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:18.152512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:18.152512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:18.152512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:18.152512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:18.152512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:18.152512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:18.152512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:18.152512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:18.152512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:18.152512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:18.152512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:18.154975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 77/162] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:02:18.155987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bd9e4-8746-4a2c-b14b-51e26d7421a0 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:18.164427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:18.207350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bfd27-11cb-4cb6-b337-913354e31185 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:18.283706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcbbee-b00a-45f5-82ab-63139487791c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:18.284788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcbbee-b00a-45f5-82ab-63139487791c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:18.285262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcbbee-b00a-45f5-82ab-63139487791c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:18.285707 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcbbee-b00a-45f5-82ab-63139487791c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:18.321912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcbbee-b00a-45f5-82ab-63139487791c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-f1dcbbee-b00a-45f5-82ab-63139487791c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:18.327134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:18.328092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:18.328862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:18.328862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:18.328862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:18.328862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:18.328862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:18.328862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:18.328862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:18.328862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:18.328862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:18.328862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:18.328862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:18.328862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:18.333190 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 86/163] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:02:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:18.335393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcbbee-b00a-45f5-82ab-63139487791c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:18.394080 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bd9e4-8746-4a2c-b14b-51e26d7421a0 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:18.445914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f3c3b-96a3-4763-9669-65a9361137b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:18.446536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f3c3b-96a3-4763-9669-65a9361137b4 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:18.446814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f3c3b-96a3-4763-9669-65a9361137b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:18.447084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f3c3b-96a3-4763-9669-65a9361137b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:18.448908 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f3c3b-96a3-4763-9669-65a9361137b4 None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=8a6d5bbd6e874220b0d317fa4794df82, user.id=56560cd65ed74a0a9e8aff03655fb0e8) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:18.449223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f3c3b-96a3-4763-9669-65a9361137b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:18.450710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f3c3b-96a3-4763-9669-65a9361137b4 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:18.461082 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 78/164] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:18 2022] GET /identity/v3/role_assignments?user.id=56560cd65ed74a0a9e8aff03655fb0e8&scope.project.id=8a6d5bbd6e874220b0d317fa4794df82&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 608 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:18.461881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f3c3b-96a3-4763-9669-65a9361137b4 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:18.550223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:18.580057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcbbee-b00a-45f5-82ab-63139487791c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:18.646261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:18.655319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2a9ab-2340-46bc-80da-a8f656ca1eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:18.656159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2a9ab-2340-46bc-80da-a8f656ca1eeb None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:18.656436 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2a9ab-2340-46bc-80da-a8f656ca1eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:18.656702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2a9ab-2340-46bc-80da-a8f656ca1eeb None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:18.657381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2a9ab-2340-46bc-80da-a8f656ca1eeb None admin] RBAC: Adding query filter params (name=member) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:18.658167 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2a9ab-2340-46bc-80da-a8f656ca1eeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:18.659903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2a9ab-2340-46bc-80da-a8f656ca1eeb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:18.670639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f3c3b-96a3-4763-9669-65a9361137b4 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:18.676939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 87/165] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:18.677626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2a9ab-2340-46bc-80da-a8f656ca1eeb None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:18.731567 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da84245-62eb-4397-8731-2b0b90736bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:18.732715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da84245-62eb-4397-8731-2b0b90736bed None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:18.733439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da84245-62eb-4397-8731-2b0b90736bed None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:18.734707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da84245-62eb-4397-8731-2b0b90736bed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:18.758223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da84245-62eb-4397-8731-2b0b90736bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-8da84245-62eb-4397-8731-2b0b90736bed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:18.769281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:18.770333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:18.771315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:18.771315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:18.771315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:18.771315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:18.771315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:18.771315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:18.771315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:18.771315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:18.771315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:18.771315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:18.771315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:18.771315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:18.773559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 79/166] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:18.774184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da84245-62eb-4397-8731-2b0b90736bed None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:18.915781 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2a9ab-2340-46bc-80da-a8f656ca1eeb None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:18.975472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da84245-62eb-4397-8731-2b0b90736bed None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:19.000387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:19.022773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75b79b-3c67-49a4-80f8-8aefda9f6f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:19.023405 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75b79b-3c67-49a4-80f8-8aefda9f6f94 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:19.023681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75b79b-3c67-49a4-80f8-8aefda9f6f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:19.023953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75b79b-3c67-49a4-80f8-8aefda9f6f94 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:19.024651 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75b79b-3c67-49a4-80f8-8aefda9f6f94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:19.024950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75b79b-3c67-49a4-80f8-8aefda9f6f94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:19.026344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75b79b-3c67-49a4-80f8-8aefda9f6f94 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:19.037468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3021f97-8739-43a8-a187-e370d708e47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:19.038106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3021f97-8739-43a8-a187-e370d708e47d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:19.038377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3021f97-8739-43a8-a187-e370d708e47d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:19.038645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3021f97-8739-43a8-a187-e370d708e47d None admin] PATH_INFO: `/v3/users/1370863b9efe4da78c3d0201098c91b5` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:19.040096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3021f97-8739-43a8-a187-e370d708e47d None admin] RBAC: Authorizing `identity:get_user(user_id=1370863b9efe4da78c3d0201098c91b5)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:19.041530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3021f97-8739-43a8-a187-e370d708e47d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:19.043270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 80/167] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:18 2022] GET /identity/v3/users/1370863b9efe4da78c3d0201098c91b5 => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:19.043849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3021f97-8739-43a8-a187-e370d708e47d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:19.050401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 88/168] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:02:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:19.057006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75b79b-3c67-49a4-80f8-8aefda9f6f94 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:19.197616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:19.208154 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3021f97-8739-43a8-a187-e370d708e47d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:19.261420 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75b79b-3c67-49a4-80f8-8aefda9f6f94 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:19.304192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52b714-1342-4fb4-8440-0f1684cd1b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:19.305941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52b714-1342-4fb4-8440-0f1684cd1b6b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:19.306635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52b714-1342-4fb4-8440-0f1684cd1b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:19.307525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52b714-1342-4fb4-8440-0f1684cd1b6b None admin] PATH_INFO: `/v3/users/1370863b9efe4da78c3d0201098c91b5` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:19.309200 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52b714-1342-4fb4-8440-0f1684cd1b6b None admin] RBAC: Authorizing `identity:get_user(user_id=1370863b9efe4da78c3d0201098c91b5)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:19.310952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52b714-1342-4fb4-8440-0f1684cd1b6b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:19.313580 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 89/169] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:19 2022] GET /identity/v3/users/1370863b9efe4da78c3d0201098c91b5 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:19.313694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e8393-2ce5-4f1c-97ea-4f4d81c182c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:19.314892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e8393-2ce5-4f1c-97ea-4f4d81c182c2 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:19.315390 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e8393-2ce5-4f1c-97ea-4f4d81c182c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:19.315764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e8393-2ce5-4f1c-97ea-4f4d81c182c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:19.316162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52b714-1342-4fb4-8440-0f1684cd1b6b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:19.316489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e8393-2ce5-4f1c-97ea-4f4d81c182c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:19.316796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e8393-2ce5-4f1c-97ea-4f4d81c182c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:19.318197 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e8393-2ce5-4f1c-97ea-4f4d81c182c2 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:19.340063 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 81/170] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:19.356803 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e8393-2ce5-4f1c-97ea-4f4d81c182c2 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:19.428437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:19.432503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52b714-1342-4fb4-8440-0f1684cd1b6b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:19.458862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:19.471614 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fae4aa-0b00-46b5-8a6d-ef93c1f59d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:19.472614 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fae4aa-0b00-46b5-8a6d-ef93c1f59d81 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:19.473090 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fae4aa-0b00-46b5-8a6d-ef93c1f59d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:19.473574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fae4aa-0b00-46b5-8a6d-ef93c1f59d81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:19.535637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e8393-2ce5-4f1c-97ea-4f4d81c182c2 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:19.547263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fae4aa-0b00-46b5-8a6d-ef93c1f59d81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:19.549662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fae4aa-0b00-46b5-8a6d-ef93c1f59d81 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:19.604721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fb62e-2cc6-4119-aecb-827faa09885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:19.605539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fb62e-2cc6-4119-aecb-827faa09885d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:19.605648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fb62e-2cc6-4119-aecb-827faa09885d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:19.605893 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fb62e-2cc6-4119-aecb-827faa09885d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:19.665513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 90/171] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:19.667214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fae4aa-0b00-46b5-8a6d-ef93c1f59d81 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:19.697781 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:19.784474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fae4aa-0b00-46b5-8a6d-ef93c1f59d81 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:19.850170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3bbff-5865-4afb-a323-05a78395f132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:19.851668 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3bbff-5865-4afb-a323-05a78395f132 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:19.852198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3bbff-5865-4afb-a323-05a78395f132 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:19.852689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3bbff-5865-4afb-a323-05a78395f132 None admin] PATH_INFO: `/v3/users/1370863b9efe4da78c3d0201098c91b5` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:19.857074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3bbff-5865-4afb-a323-05a78395f132 None admin] RBAC: Authorizing `identity:get_user(user_id=1370863b9efe4da78c3d0201098c91b5)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:19.858855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3bbff-5865-4afb-a323-05a78395f132 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:19.861366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 91/172] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:19 2022] GET /identity/v3/users/1370863b9efe4da78c3d0201098c91b5 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:19.862396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3bbff-5865-4afb-a323-05a78395f132 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:19.904978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6fb62e-2cc6-4119-aecb-827faa09885d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:19.906507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6fb62e-2cc6-4119-aecb-827faa09885d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:19.994942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:19.998880 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3bbff-5865-4afb-a323-05a78395f132 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:20.002541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:20.039690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b6588-db9b-448d-a583-7350de251013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:20.040618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b6588-db9b-448d-a583-7350de251013 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.040997 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b6588-db9b-448d-a583-7350de251013 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.041367 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b6588-db9b-448d-a583-7350de251013 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.042694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b6588-db9b-448d-a583-7350de251013 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:20.044301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b6588-db9b-448d-a583-7350de251013 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:20.046716 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 92/173] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:19 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:20.047649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b6588-db9b-448d-a583-7350de251013 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:20.192628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 82/174] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 847 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:20.193867 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fb62e-2cc6-4119-aecb-827faa09885d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:20.252381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b6588-db9b-448d-a583-7350de251013 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:20.321202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fc33c-3caf-4a03-83f2-e64dfa90d551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:20.322112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fc33c-3caf-4a03-83f2-e64dfa90d551 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.322606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fc33c-3caf-4a03-83f2-e64dfa90d551 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.323016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fc33c-3caf-4a03-83f2-e64dfa90d551 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.381660 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:20.398061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fb62e-2cc6-4119-aecb-827faa09885d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:20.451371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e7d38-c1b6-40af-979e-f89cd4c835c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:20.452124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e7d38-c1b6-40af-979e-f89cd4c835c8 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.452644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e7d38-c1b6-40af-979e-f89cd4c835c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.453392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e7d38-c1b6-40af-979e-f89cd4c835c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.454396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e7d38-c1b6-40af-979e-f89cd4c835c8 None admin] RBAC: Adding query filter params (role.id=b07ea738cc204b2b8125bf4cd6d9c54a, scope.project.id=8a6d5bbd6e874220b0d317fa4794df82, user.id=1370863b9efe4da78c3d0201098c91b5) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:20.455485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e7d38-c1b6-40af-979e-f89cd4c835c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:20.458887 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e7d38-c1b6-40af-979e-f89cd4c835c8 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:20.460798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:20.468490 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 83/175] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:20 2022] GET /identity/v3/role_assignments?user.id=1370863b9efe4da78c3d0201098c91b5&scope.project.id=8a6d5bbd6e874220b0d317fa4794df82&role.id=b07ea738cc204b2b8125bf4cd6d9c54a => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:20.471723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4d86a-4f08-4873-bf64-4459c2a1a2a5 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.471723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4d86a-4f08-4873-bf64-4459c2a1a2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.471723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4d86a-4f08-4873-bf64-4459c2a1a2a5 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.472866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 84/176] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:20.478479 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3fc33c-3caf-4a03-83f2-e64dfa90d551 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:20.478683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4d86a-4f08-4873-bf64-4459c2a1a2a5 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:20.480383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3fc33c-3caf-4a03-83f2-e64dfa90d551 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:20.678025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 93/177] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:20.679327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdde873-c034-4be0-b411-41c525892af7 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.679610 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdde873-c034-4be0-b411-41c525892af7 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.679959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdde873-c034-4be0-b411-41c525892af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.693012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4d86a-4f08-4873-bf64-4459c2a1a2a5 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:20.705070 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6fdde873-c034-4be0-b411-41c525892af7 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:20.707211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6fdde873-c034-4be0-b411-41c525892af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:20.736831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e773cc4-de77-4c99-b5f5-6cfcd36b3657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:20.737540 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e773cc4-de77-4c99-b5f5-6cfcd36b3657 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.737746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e773cc4-de77-4c99-b5f5-6cfcd36b3657 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.738015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e773cc4-de77-4c99-b5f5-6cfcd36b3657 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.739175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e773cc4-de77-4c99-b5f5-6cfcd36b3657 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:20.740709 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e773cc4-de77-4c99-b5f5-6cfcd36b3657 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:20.742408 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 85/178] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:20 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:20.743621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592f19c-1465-4c0b-bcbb-bcc13a8b7e39 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.743966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592f19c-1465-4c0b-bcbb-bcc13a8b7e39 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.744233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592f19c-1465-4c0b-bcbb-bcc13a8b7e39 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.745472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 86/179] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:20.746472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be097c-c12e-4c4e-bcfa-4512dfc22e1c None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.746743 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be097c-c12e-4c4e-bcfa-4512dfc22e1c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.747007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be097c-c12e-4c4e-bcfa-4512dfc22e1c None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.748362 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 87/180] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:20.749244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6dbdf-0869-4046-a5e0-94e8d3711625 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.749508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6dbdf-0869-4046-a5e0-94e8d3711625 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.749775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6dbdf-0869-4046-a5e0-94e8d3711625 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.751177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 88/181] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:20.751806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6dbdf-0869-4046-a5e0-94e8d3711625 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:20.782319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:20.796386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 94/182] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:20.798389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdde873-c034-4be0-b411-41c525892af7 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:20.833343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:20.870848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6dbdf-0869-4046-a5e0-94e8d3711625 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:20.910886 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdde873-c034-4be0-b411-41c525892af7 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:20.924471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d19bda-44a2-4950-b3e2-a822684461b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:20.925260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d19bda-44a2-4950-b3e2-a822684461b4 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.925541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d19bda-44a2-4950-b3e2-a822684461b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.925816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d19bda-44a2-4950-b3e2-a822684461b4 None admin] PATH_INFO: `/v3/projects/962aa8a1586e4f29a0e5a49b25c662ba` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.927231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d19bda-44a2-4950-b3e2-a822684461b4 None admin] RBAC: Authorizing `identity:get_project(project_id=962aa8a1586e4f29a0e5a49b25c662ba)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:20.928727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d19bda-44a2-4950-b3e2-a822684461b4 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:20.930611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 89/183] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:20 2022] GET /identity/v3/projects/962aa8a1586e4f29a0e5a49b25c662ba => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:20.931651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece265c-a0fe-406b-be1e-79862b9dd1f0 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.931955 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece265c-a0fe-406b-be1e-79862b9dd1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.932270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece265c-a0fe-406b-be1e-79862b9dd1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.953127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52071ef3-b8a5-4dfc-a22d-54bacb59c52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:20.953748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52071ef3-b8a5-4dfc-a22d-54bacb59c52f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.954044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52071ef3-b8a5-4dfc-a22d-54bacb59c52f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.954329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52071ef3-b8a5-4dfc-a22d-54bacb59c52f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.956122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8ece265c-a0fe-406b-be1e-79862b9dd1f0 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:20.956252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52071ef3-b8a5-4dfc-a22d-54bacb59c52f None admin] RBAC: Adding query filter params (role.id=7562948bd63d483cad68427819d434a9, scope.project.id=8a6d5bbd6e874220b0d317fa4794df82, user.id=1370863b9efe4da78c3d0201098c91b5) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:20.956252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52071ef3-b8a5-4dfc-a22d-54bacb59c52f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:20.957350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8ece265c-a0fe-406b-be1e-79862b9dd1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:20.957545 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52071ef3-b8a5-4dfc-a22d-54bacb59c52f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:20.976392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 95/184] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:20 2022] GET /identity/v3/role_assignments?user.id=1370863b9efe4da78c3d0201098c91b5&scope.project.id=8a6d5bbd6e874220b0d317fa4794df82&role.id=7562948bd63d483cad68427819d434a9 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:20.976392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81ed6c-e1c1-41dc-9ff6-bd5670d0a7a9 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:20.976392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81ed6c-e1c1-41dc-9ff6-bd5670d0a7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:20.976392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81ed6c-e1c1-41dc-9ff6-bd5670d0a7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:20.994761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7c81ed6c-e1c1-41dc-9ff6-bd5670d0a7a9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:20.996142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7c81ed6c-e1c1-41dc-9ff6-bd5670d0a7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:21.036831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 90/185] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:21.038049 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a4cb0-6221-434d-95c0-73fa135c6f58 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:21.038427 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a4cb0-6221-434d-95c0-73fa135c6f58 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:21.038812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a4cb0-6221-434d-95c0-73fa135c6f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:21.073611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:21.089125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d77a4cb0-6221-434d-95c0-73fa135c6f58 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:21.089935 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 96/186] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:21.090338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d77a4cb0-6221-434d-95c0-73fa135c6f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:21.090980 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81ed6c-e1c1-41dc-9ff6-bd5670d0a7a9 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:21.163828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:21.174931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 91/187] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:21.175755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a4cb0-6221-434d-95c0-73fa135c6f58 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:21.209370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81ed6c-e1c1-41dc-9ff6-bd5670d0a7a9 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:21.246294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde6390-cafd-4f94-90ac-d62777ed14d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:21.246917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde6390-cafd-4f94-90ac-d62777ed14d1 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:21.247199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde6390-cafd-4f94-90ac-d62777ed14d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:21.247473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde6390-cafd-4f94-90ac-d62777ed14d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:21.249160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde6390-cafd-4f94-90ac-d62777ed14d1 None admin] RBAC: Adding query filter params (role.id=b07ea738cc204b2b8125bf4cd6d9c54a, scope.project.id=962aa8a1586e4f29a0e5a49b25c662ba, user.id=1370863b9efe4da78c3d0201098c91b5) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:21.249565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde6390-cafd-4f94-90ac-d62777ed14d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:21.251119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde6390-cafd-4f94-90ac-d62777ed14d1 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:21.258707 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 97/188] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:21 2022] GET /identity/v3/role_assignments?user.id=1370863b9efe4da78c3d0201098c91b5&scope.project.id=962aa8a1586e4f29a0e5a49b25c662ba&role.id=b07ea738cc204b2b8125bf4cd6d9c54a => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:21.375764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde6390-cafd-4f94-90ac-d62777ed14d1 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:21.375948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a4cb0-6221-434d-95c0-73fa135c6f58 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:21.434938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8e6eb-fd1d-4306-a0ea-d69f1b2df48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:21.435605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8e6eb-fd1d-4306-a0ea-d69f1b2df48b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:21.435884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8e6eb-fd1d-4306-a0ea-d69f1b2df48b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:21.436237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8e6eb-fd1d-4306-a0ea-d69f1b2df48b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:21.456495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8e6eb-fd1d-4306-a0ea-d69f1b2df48b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:21.462020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-6dd8e6eb-fd1d-4306-a0ea-d69f1b2df48b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:21.462700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:21.463615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:21.464557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 92/189] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:02:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:21.465208 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8e6eb-fd1d-4306-a0ea-d69f1b2df48b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:21.547570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:21.553355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde6390-cafd-4f94-90ac-d62777ed14d1 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:21.599269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fab33b-4ac2-48cd-b307-1e616890215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:21.600052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fab33b-4ac2-48cd-b307-1e616890215a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:21.600329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fab33b-4ac2-48cd-b307-1e616890215a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:21.600605 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fab33b-4ac2-48cd-b307-1e616890215a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:21.601760 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fab33b-4ac2-48cd-b307-1e616890215a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:21.603236 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fab33b-4ac2-48cd-b307-1e616890215a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:21.604889 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 98/190] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:02:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:21.605674 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fab33b-4ac2-48cd-b307-1e616890215a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:21.617262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8e6eb-fd1d-4306-a0ea-d69f1b2df48b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:21.661713 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad482d-d11b-4cd8-bcf2-d451c01a9655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:21.662637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad482d-d11b-4cd8-bcf2-d451c01a9655 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:21.663051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad482d-d11b-4cd8-bcf2-d451c01a9655 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:21.663455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad482d-d11b-4cd8-bcf2-d451c01a9655 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:21.675732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad482d-d11b-4cd8-bcf2-d451c01a9655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-36ad482d-d11b-4cd8-bcf2-d451c01a9655 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:21.680735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:21.681612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:21.682387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 93/191] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:21.683054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad482d-d11b-4cd8-bcf2-d451c01a9655 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:21.720304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fab33b-4ac2-48cd-b307-1e616890215a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:21.737418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:21.757340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06442b9d-71a7-4dec-bc20-46ae2a01f5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:21.757953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06442b9d-71a7-4dec-bc20-46ae2a01f5cc None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:21.758235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06442b9d-71a7-4dec-bc20-46ae2a01f5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:21.758557 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06442b9d-71a7-4dec-bc20-46ae2a01f5cc None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:21.759228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06442b9d-71a7-4dec-bc20-46ae2a01f5cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:21.759531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06442b9d-71a7-4dec-bc20-46ae2a01f5cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:21.760948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06442b9d-71a7-4dec-bc20-46ae2a01f5cc None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:21.771150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 99/192] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:21.772865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06442b9d-71a7-4dec-bc20-46ae2a01f5cc None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:21.787173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad482d-d11b-4cd8-bcf2-d451c01a9655 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:21.806045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:21.828018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415b3a4-3bde-4b56-8776-6aeb3f86c465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:21.828676 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415b3a4-3bde-4b56-8776-6aeb3f86c465 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:21.828954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415b3a4-3bde-4b56-8776-6aeb3f86c465 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:21.829226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415b3a4-3bde-4b56-8776-6aeb3f86c465 None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:21.830016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415b3a4-3bde-4b56-8776-6aeb3f86c465 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:21.832192 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415b3a4-3bde-4b56-8776-6aeb3f86c465 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:21.856118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 94/193] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:02:21 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:21.857356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415b3a4-3bde-4b56-8776-6aeb3f86c465 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:21.899814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06442b9d-71a7-4dec-bc20-46ae2a01f5cc None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:21.944740 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d4366-797d-4f3c-96be-79fd5ae2247d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:21.945409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d4366-797d-4f3c-96be-79fd5ae2247d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:21.945874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d4366-797d-4f3c-96be-79fd5ae2247d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:21.946156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d4366-797d-4f3c-96be-79fd5ae2247d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:21.960671 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d4366-797d-4f3c-96be-79fd5ae2247d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-593d4366-797d-4f3c-96be-79fd5ae2247d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:21.966349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:21.967326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 100/194] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:02:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:21.968622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d4366-797d-4f3c-96be-79fd5ae2247d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:21.997062 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415b3a4-3bde-4b56-8776-6aeb3f86c465 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:22.031669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:22.045536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15b830-ba8c-430a-8eb7-b3ca96f8e236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:22.046174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15b830-ba8c-430a-8eb7-b3ca96f8e236 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:22.046456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15b830-ba8c-430a-8eb7-b3ca96f8e236 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:22.046766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15b830-ba8c-430a-8eb7-b3ca96f8e236 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:22.047448 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15b830-ba8c-430a-8eb7-b3ca96f8e236 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:22.047754 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15b830-ba8c-430a-8eb7-b3ca96f8e236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:22.049934 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15b830-ba8c-430a-8eb7-b3ca96f8e236 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:22.059942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 95/195] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:22.060872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15b830-ba8c-430a-8eb7-b3ca96f8e236 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:22.107448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d4366-797d-4f3c-96be-79fd5ae2247d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:22.115280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:22.152464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98338616-24c2-4c10-98e7-6d0be3585308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:22.153113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98338616-24c2-4c10-98e7-6d0be3585308 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:22.153390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98338616-24c2-4c10-98e7-6d0be3585308 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:22.153657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98338616-24c2-4c10-98e7-6d0be3585308 None admin] PATH_INFO: `/v3/users/42862c6f25c1440eab3b05c40bdece91` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:22.176793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98338616-24c2-4c10-98e7-6d0be3585308 None admin] RBAC: Authorizing `identity:get_user(user_id=42862c6f25c1440eab3b05c40bdece91)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:22.178319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98338616-24c2-4c10-98e7-6d0be3585308 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:22.180773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 101/196] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:21 2022] GET /identity/v3/users/42862c6f25c1440eab3b05c40bdece91 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:22.181487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98338616-24c2-4c10-98e7-6d0be3585308 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:22.217759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15b830-ba8c-430a-8eb7-b3ca96f8e236 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:22.263942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42390eef-96a1-497e-8595-c05c71ed28ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:22.264976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42390eef-96a1-497e-8595-c05c71ed28ac None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:22.265331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42390eef-96a1-497e-8595-c05c71ed28ac None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:22.265652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42390eef-96a1-497e-8595-c05c71ed28ac None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:22.266361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42390eef-96a1-497e-8595-c05c71ed28ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:22.266635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42390eef-96a1-497e-8595-c05c71ed28ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:22.269064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42390eef-96a1-497e-8595-c05c71ed28ac None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:22.279652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 96/197] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:02:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:22.280533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42390eef-96a1-497e-8595-c05c71ed28ac None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:22.310433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:22.321529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98338616-24c2-4c10-98e7-6d0be3585308 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:22.381200 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2cb13-0c2f-46ec-8d9c-e3685714e849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:22.382193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2cb13-0c2f-46ec-8d9c-e3685714e849 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:22.382978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2cb13-0c2f-46ec-8d9c-e3685714e849 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:22.383254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2cb13-0c2f-46ec-8d9c-e3685714e849 None admin] PATH_INFO: `/v3/users/c5c61e97018d4a15825261fec8237dad` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:22.384754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2cb13-0c2f-46ec-8d9c-e3685714e849 None admin] RBAC: Authorizing `identity:get_user(user_id=c5c61e97018d4a15825261fec8237dad)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:22.387105 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2cb13-0c2f-46ec-8d9c-e3685714e849 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:22.388804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 102/198] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:22 2022] GET /identity/v3/users/c5c61e97018d4a15825261fec8237dad => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:22.389574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2cb13-0c2f-46ec-8d9c-e3685714e849 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:22.407458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:22.415724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42390eef-96a1-497e-8595-c05c71ed28ac None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:22.460711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a17e89-cb5e-468e-a347-a449f1452f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:22.461588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a17e89-cb5e-468e-a347-a449f1452f96 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:22.462046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a17e89-cb5e-468e-a347-a449f1452f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:22.462548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a17e89-cb5e-468e-a347-a449f1452f96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:22.529516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2cb13-0c2f-46ec-8d9c-e3685714e849 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:22.550652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a17e89-cb5e-468e-a347-a449f1452f96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:22.552932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a17e89-cb5e-468e-a347-a449f1452f96 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:22.576936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919c5c2-1915-40b2-96cb-3fef0c12d02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:22.577727 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919c5c2-1915-40b2-96cb-3fef0c12d02a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:22.578106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919c5c2-1915-40b2-96cb-3fef0c12d02a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:22.578474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919c5c2-1915-40b2-96cb-3fef0c12d02a None admin] PATH_INFO: `/v3/users/42862c6f25c1440eab3b05c40bdece91` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:22.580904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919c5c2-1915-40b2-96cb-3fef0c12d02a None admin] RBAC: Authorizing `identity:get_user(user_id=42862c6f25c1440eab3b05c40bdece91)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:22.583187 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919c5c2-1915-40b2-96cb-3fef0c12d02a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:22.586328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 103/199] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:22 2022] GET /identity/v3/users/42862c6f25c1440eab3b05c40bdece91 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:22.588033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919c5c2-1915-40b2-96cb-3fef0c12d02a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:22.636961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:22.669658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:22.680299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 97/200] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:22.681381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a17e89-cb5e-468e-a347-a449f1452f96 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:22.745790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919c5c2-1915-40b2-96cb-3fef0c12d02a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:22.806072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71881472-6f3a-4376-9afb-913da5f2a870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:22.806609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71881472-6f3a-4376-9afb-913da5f2a870 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:22.806882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71881472-6f3a-4376-9afb-913da5f2a870 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:22.807148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71881472-6f3a-4376-9afb-913da5f2a870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:22.807453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a17e89-cb5e-468e-a347-a449f1452f96 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:22.846935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17620213-7924-42b4-a6cd-cef7dbe7fb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:22.847699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17620213-7924-42b4-a6cd-cef7dbe7fb8c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:22.848114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17620213-7924-42b4-a6cd-cef7dbe7fb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:22.848523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17620213-7924-42b4-a6cd-cef7dbe7fb8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:22.888435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71881472-6f3a-4376-9afb-913da5f2a870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:22.890268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71881472-6f3a-4376-9afb-913da5f2a870 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:22.934970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:22.936403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:22.945369 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17620213-7924-42b4-a6cd-cef7dbe7fb8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:22.947337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17620213-7924-42b4-a6cd-cef7dbe7fb8c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:23.027595 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 104/201] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:23.028612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71881472-6f3a-4376-9afb-913da5f2a870 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:23.059416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 98/202] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:23.060102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17620213-7924-42b4-a6cd-cef7dbe7fb8c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:23.168057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:23.174025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17620213-7924-42b4-a6cd-cef7dbe7fb8c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:23.181359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71881472-6f3a-4376-9afb-913da5f2a870 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:23.219655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cea98f-939f-41fa-b69b-31cba5adfa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:23.221316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cea98f-939f-41fa-b69b-31cba5adfa94 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:23.221607 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cea98f-939f-41fa-b69b-31cba5adfa94 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:23.222020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cea98f-939f-41fa-b69b-31cba5adfa94 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:23.223321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cea98f-939f-41fa-b69b-31cba5adfa94 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:23.224404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:23.225105 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cea98f-939f-41fa-b69b-31cba5adfa94 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:23.227976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 99/203] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:23 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:23.228792 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cea98f-939f-41fa-b69b-31cba5adfa94 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:23.231269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd8c1a-41ed-4591-aa41-6faa86612094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:23.232114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd8c1a-41ed-4591-aa41-6faa86612094 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:23.232529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd8c1a-41ed-4591-aa41-6faa86612094 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:23.232934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd8c1a-41ed-4591-aa41-6faa86612094 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:23.234484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd8c1a-41ed-4591-aa41-6faa86612094 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:23.237011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd8c1a-41ed-4591-aa41-6faa86612094 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:23.239885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 105/204] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:23 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:23.240939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd8c1a-41ed-4591-aa41-6faa86612094 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:23.366673 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cea98f-939f-41fa-b69b-31cba5adfa94 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:23.382260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd8c1a-41ed-4591-aa41-6faa86612094 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:23.405232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60fd5c-91b4-4aae-8261-186a8554a67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:23.406525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60fd5c-91b4-4aae-8261-186a8554a67c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:23.406999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60fd5c-91b4-4aae-8261-186a8554a67c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:23.407463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60fd5c-91b4-4aae-8261-186a8554a67c None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:23.409531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60fd5c-91b4-4aae-8261-186a8554a67c None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:23.411941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60fd5c-91b4-4aae-8261-186a8554a67c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:23.415127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 100/205] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:23 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:23.416158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60fd5c-91b4-4aae-8261-186a8554a67c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:23.433886 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014f7cf-6eb1-4306-b448-d2214e9d5c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:23.434786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014f7cf-6eb1-4306-b448-d2214e9d5c2b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:23.435064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014f7cf-6eb1-4306-b448-d2214e9d5c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:23.435329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014f7cf-6eb1-4306-b448-d2214e9d5c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:23.436578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014f7cf-6eb1-4306-b448-d2214e9d5c2b None admin] RBAC: Adding query filter params (role.id=cd6b754c94e54de6b3bc5e17618f1e67, scope.project.id=8a6d5bbd6e874220b0d317fa4794df82, user.id=c5c61e97018d4a15825261fec8237dad) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:23.436898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014f7cf-6eb1-4306-b448-d2214e9d5c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:23.438645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014f7cf-6eb1-4306-b448-d2214e9d5c2b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:23.446457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 106/206] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:23 2022] GET /identity/v3/role_assignments?user.id=c5c61e97018d4a15825261fec8237dad&scope.project.id=8a6d5bbd6e874220b0d317fa4794df82&role.id=cd6b754c94e54de6b3bc5e17618f1e67 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:23.448748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014f7cf-6eb1-4306-b448-d2214e9d5c2b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:23.471556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:23.571067 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:23.599134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60fd5c-91b4-4aae-8261-186a8554a67c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:23.641367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93285fd8-56b7-4932-bff3-11e7068c0ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:23.643299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93285fd8-56b7-4932-bff3-11e7068c0ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:23.643798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93285fd8-56b7-4932-bff3-11e7068c0ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:23.644392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93285fd8-56b7-4932-bff3-11e7068c0ee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:23.645463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93285fd8-56b7-4932-bff3-11e7068c0ee1 None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=dcbc59eca63d4c3ea02d6e944948c888, user.id=42862c6f25c1440eab3b05c40bdece91) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:23.645964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93285fd8-56b7-4932-bff3-11e7068c0ee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:23.647602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93285fd8-56b7-4932-bff3-11e7068c0ee1 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:23.648161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014f7cf-6eb1-4306-b448-d2214e9d5c2b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:23.654350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 101/207] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:23 2022] GET /identity/v3/role_assignments?user.id=42862c6f25c1440eab3b05c40bdece91&scope.project.id=dcbc59eca63d4c3ea02d6e944948c888&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:23.702295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca4d12-25fb-4df9-a27f-f48c6d96430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:23.702941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca4d12-25fb-4df9-a27f-f48c6d96430e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:23.703221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca4d12-25fb-4df9-a27f-f48c6d96430e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:23.703499 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca4d12-25fb-4df9-a27f-f48c6d96430e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:23.704526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca4d12-25fb-4df9-a27f-f48c6d96430e None admin] RBAC: Adding query filter params (role.id=7562948bd63d483cad68427819d434a9, scope.project.id=dcbc59eca63d4c3ea02d6e944948c888, user.id=42862c6f25c1440eab3b05c40bdece91) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:23.704963 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca4d12-25fb-4df9-a27f-f48c6d96430e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:23.706999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca4d12-25fb-4df9-a27f-f48c6d96430e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:23.714019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 107/208] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:23 2022] GET /identity/v3/role_assignments?user.id=42862c6f25c1440eab3b05c40bdece91&scope.project.id=dcbc59eca63d4c3ea02d6e944948c888&role.id=7562948bd63d483cad68427819d434a9 => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:24.048138 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4654fb-6ed3-444a-aed2-d658cbdc865c None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:24.048876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4654fb-6ed3-444a-aed2-d658cbdc865c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:24.049347 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4654fb-6ed3-444a-aed2-d658cbdc865c None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:24.050895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 102/209] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:24.056691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2688d-ac83-4850-b5ec-da81ffa8b90b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:24.057158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2688d-ac83-4850-b5ec-da81ffa8b90b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:24.057255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2688d-ac83-4850-b5ec-da81ffa8b90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:24.076815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-0aa2688d-ac83-4850-b5ec-da81ffa8b90b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:24.078015 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0aa2688d-ac83-4850-b5ec-da81ffa8b90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:24.165819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 108/210] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:24.208941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b249dde-df57-48d4-a906-4cc29193f3de None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:24.209478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b249dde-df57-48d4-a906-4cc29193f3de None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:24.209701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b249dde-df57-48d4-a906-4cc29193f3de None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:24.211815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 103/211] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:24.218668 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a4382-d17c-410b-8e60-615c703f8a1f None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:24.218955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a4382-d17c-410b-8e60-615c703f8a1f None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:24.219228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a4382-d17c-410b-8e60-615c703f8a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:24.247244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-bb9a4382-d17c-410b-8e60-615c703f8a1f None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:24.249380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-bb9a4382-d17c-410b-8e60-615c703f8a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:24.326113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:24.341910 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 109/212] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:24.452608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b249dde-df57-48d4-a906-4cc29193f3de None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:24.557757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:24.566579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b249dde-df57-48d4-a906-4cc29193f3de None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:24.607159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c84ee3-e108-4a96-94fb-1686b8601c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:24.607709 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c84ee3-e108-4a96-94fb-1686b8601c65 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:24.607983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c84ee3-e108-4a96-94fb-1686b8601c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:24.608279 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c84ee3-e108-4a96-94fb-1686b8601c65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:24.619655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c84ee3-e108-4a96-94fb-1686b8601c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-25c84ee3-e108-4a96-94fb-1686b8601c65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:24.624565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:24.625443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:24.626233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 104/213] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:24.627438 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a4382-d17c-410b-8e60-615c703f8a1f None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:24.633424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c84ee3-e108-4a96-94fb-1686b8601c65 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:24.744333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c84ee3-e108-4a96-94fb-1686b8601c65 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:24.749618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a4382-d17c-410b-8e60-615c703f8a1f None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:24.790454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a235c6-433f-46a3-8c6c-df25720eccb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:24.790454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a235c6-433f-46a3-8c6c-df25720eccb4 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:24.790845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a235c6-433f-46a3-8c6c-df25720eccb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:24.790845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a235c6-433f-46a3-8c6c-df25720eccb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:24.792012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a235c6-433f-46a3-8c6c-df25720eccb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:24.793483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a235c6-433f-46a3-8c6c-df25720eccb4 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:24.795618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 110/214] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:02:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:24.801450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a235c6-433f-46a3-8c6c-df25720eccb4 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:24.802191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99798a2-ac69-4bf6-8ab8-89b013e46050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:24.803032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99798a2-ac69-4bf6-8ab8-89b013e46050 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:24.803522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99798a2-ac69-4bf6-8ab8-89b013e46050 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:24.803982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99798a2-ac69-4bf6-8ab8-89b013e46050 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:24.805018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99798a2-ac69-4bf6-8ab8-89b013e46050 None admin] RBAC: Adding query filter params (name=member) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:24.805327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99798a2-ac69-4bf6-8ab8-89b013e46050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:24.806708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99798a2-ac69-4bf6-8ab8-89b013e46050 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:24.820280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 105/215] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:24.822640 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99798a2-ac69-4bf6-8ab8-89b013e46050 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:24.844245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:24.884614 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:24.941892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99798a2-ac69-4bf6-8ab8-89b013e46050 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:24.949551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a235c6-433f-46a3-8c6c-df25720eccb4 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:24.984419 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854bef1-2248-4df8-b96d-ceedddd3ae9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:24.985172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854bef1-2248-4df8-b96d-ceedddd3ae9f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:24.985455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854bef1-2248-4df8-b96d-ceedddd3ae9f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:24.985729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854bef1-2248-4df8-b96d-ceedddd3ae9f None admin] PATH_INFO: `/v3/users/f0194d28c9454e32b4d3a9ea7f5e7178` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:24.999450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd8088-7282-4cd6-8db7-987da1b3cb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:25.000488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd8088-7282-4cd6-8db7-987da1b3cb48 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:25.000920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd8088-7282-4cd6-8db7-987da1b3cb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:25.001319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd8088-7282-4cd6-8db7-987da1b3cb48 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:25.003068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854bef1-2248-4df8-b96d-ceedddd3ae9f None admin] RBAC: Authorizing `identity:get_user(user_id=f0194d28c9454e32b4d3a9ea7f5e7178)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:25.003309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd8088-7282-4cd6-8db7-987da1b3cb48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:25.004828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd8088-7282-4cd6-8db7-987da1b3cb48 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:25.004945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854bef1-2248-4df8-b96d-ceedddd3ae9f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:25.006875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 106/216] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:24 2022] GET /identity/v3/users/f0194d28c9454e32b4d3a9ea7f5e7178 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:25.012817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854bef1-2248-4df8-b96d-ceedddd3ae9f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:25.031234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 111/217] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:02:24 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:25.133966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:25.162104 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854bef1-2248-4df8-b96d-ceedddd3ae9f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:25.204468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad47965-698f-4e7f-ac7d-7f1725580328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:25.205230 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad47965-698f-4e7f-ac7d-7f1725580328 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:25.205493 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad47965-698f-4e7f-ac7d-7f1725580328 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:25.205764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad47965-698f-4e7f-ac7d-7f1725580328 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:25.287745 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad47965-698f-4e7f-ac7d-7f1725580328 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:25.289363 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad47965-698f-4e7f-ac7d-7f1725580328 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:25.386421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:25.416037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 107/218] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 01:02:25.421885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd8088-7282-4cd6-8db7-987da1b3cb48 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:25.557926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd8088-7282-4cd6-8db7-987da1b3cb48 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:25.584871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:25.608417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6540813-e912-4b40-aa28-df865138c343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:25.609601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6540813-e912-4b40-aa28-df865138c343 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:25.610156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6540813-e912-4b40-aa28-df865138c343 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:25.610721 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6540813-e912-4b40-aa28-df865138c343 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:25.616941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6540813-e912-4b40-aa28-df865138c343 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:25.619216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6540813-e912-4b40-aa28-df865138c343 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:25.621749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 112/219] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:25 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:25.628019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad47965-698f-4e7f-ac7d-7f1725580328 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:25.743795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad47965-698f-4e7f-ac7d-7f1725580328 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:25.784675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731eb1e-f6e4-47d0-a772-a3f7e170a06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:25.785294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731eb1e-f6e4-47d0-a772-a3f7e170a06d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:25.785568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731eb1e-f6e4-47d0-a772-a3f7e170a06d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:25.785839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731eb1e-f6e4-47d0-a772-a3f7e170a06d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:25.786574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731eb1e-f6e4-47d0-a772-a3f7e170a06d None admin] RBAC: Adding query filter params (role.id=b07ea738cc204b2b8125bf4cd6d9c54a, scope.project.id=dcbc59eca63d4c3ea02d6e944948c888, user.id=f0194d28c9454e32b4d3a9ea7f5e7178) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:25.786880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731eb1e-f6e4-47d0-a772-a3f7e170a06d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:25.788372 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731eb1e-f6e4-47d0-a772-a3f7e170a06d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:25.794534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 108/220] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:25 2022] GET /identity/v3/role_assignments?user.id=f0194d28c9454e32b4d3a9ea7f5e7178&scope.project.id=dcbc59eca63d4c3ea02d6e944948c888&role.id=b07ea738cc204b2b8125bf4cd6d9c54a => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:25.877186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb776b-3363-4be4-8c37-a174a6d00456 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:25.877511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb776b-3363-4be4-8c37-a174a6d00456 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:25.877743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb776b-3363-4be4-8c37-a174a6d00456 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:25.879190 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 113/221] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:25.885359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbf71f-78c2-4a4d-a0ca-0d7d93907eef None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:25.885749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbf71f-78c2-4a4d-a0ca-0d7d93907eef None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:25.885923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbf71f-78c2-4a4d-a0ca-0d7d93907eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:25.897429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42e84f-627d-4084-890f-fc25b2bc42a2 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:25.897804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42e84f-627d-4084-890f-fc25b2bc42a2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:25.898011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42e84f-627d-4084-890f-fc25b2bc42a2 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:25.899527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 114/222] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:02:25.908238 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a6f71-a8f1-4d9b-8d8f-b138879128fc None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:25.908553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a6f71-a8f1-4d9b-8d8f-b138879128fc None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:25.908831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a6f71-a8f1-4d9b-8d8f-b138879128fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:25.920433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-08dbf71f-78c2-4a4d-a0ca-0d7d93907eef None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:25.921691 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-08dbf71f-78c2-4a4d-a0ca-0d7d93907eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:25.943246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-0d9a6f71-a8f1-4d9b-8d8f-b138879128fc None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:25.944569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0d9a6f71-a8f1-4d9b-8d8f-b138879128fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:26.003364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:26.052399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 115/223] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:02:26.055104 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 109/224] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:26.360681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a6f71-a8f1-4d9b-8d8f-b138879128fc None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:26.442046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbf71f-78c2-4a4d-a0ca-0d7d93907eef None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:26.468422 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a6f71-a8f1-4d9b-8d8f-b138879128fc None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:26.488368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:26.510329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed09a9-0119-46f7-83f2-e579973bdced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:26.510329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed09a9-0119-46f7-83f2-e579973bdced None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:26.510329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed09a9-0119-46f7-83f2-e579973bdced None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:26.510329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed09a9-0119-46f7-83f2-e579973bdced None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:26.523846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed09a9-0119-46f7-83f2-e579973bdced None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-7bed09a9-0119-46f7-83f2-e579973bdced None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:26.530268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:26.531237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:26.532487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 116/225] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:02:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:26.538990 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed09a9-0119-46f7-83f2-e579973bdced None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:26.659190 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed09a9-0119-46f7-83f2-e579973bdced None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:26.698291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05038269-541b-4dae-a38f-cc37c202fbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:26.698915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05038269-541b-4dae-a38f-cc37c202fbeb None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:26.699192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05038269-541b-4dae-a38f-cc37c202fbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:26.699463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05038269-541b-4dae-a38f-cc37c202fbeb None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:26.700196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05038269-541b-4dae-a38f-cc37c202fbeb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:26.700503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05038269-541b-4dae-a38f-cc37c202fbeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:26.702201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05038269-541b-4dae-a38f-cc37c202fbeb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:26.712551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 117/226] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:02:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:26.718080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05038269-541b-4dae-a38f-cc37c202fbeb None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:26.720431 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbf71f-78c2-4a4d-a0ca-0d7d93907eef None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:26.776561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:26.842042 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05038269-541b-4dae-a38f-cc37c202fbeb None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:26.864952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546210e-ab70-4fad-9809-767ca7986427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:26.866064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546210e-ab70-4fad-9809-767ca7986427 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:26.866571 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546210e-ab70-4fad-9809-767ca7986427 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:26.867038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546210e-ab70-4fad-9809-767ca7986427 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:26.878769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baea088-17a6-4b29-9bc2-68e79c353d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:26.879817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baea088-17a6-4b29-9bc2-68e79c353d1c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:26.880135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baea088-17a6-4b29-9bc2-68e79c353d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:26.880411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baea088-17a6-4b29-9bc2-68e79c353d1c None admin] PATH_INFO: `/v3/users/42862c6f25c1440eab3b05c40bdece91` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:26.881576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baea088-17a6-4b29-9bc2-68e79c353d1c None admin] RBAC: Authorizing `identity:get_user(user_id=42862c6f25c1440eab3b05c40bdece91)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:26.885155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baea088-17a6-4b29-9bc2-68e79c353d1c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:26.885155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 118/227] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:26 2022] GET /identity/v3/users/42862c6f25c1440eab3b05c40bdece91 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:26.890712 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baea088-17a6-4b29-9bc2-68e79c353d1c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:26.916757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546210e-ab70-4fad-9809-767ca7986427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-a546210e-ab70-4fad-9809-767ca7986427 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:26.922293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:26.923189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:26.924534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:26.924534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:26.924534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:26.924534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:26.924534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:26.924534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:26.924534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:26.924534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:26.924534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:26.924534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:26.924534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:26.924534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:26.927194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 110/228] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:02:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 485 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:26.932706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546210e-ab70-4fad-9809-767ca7986427 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:26.965815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:27.078807 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:27.081969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baea088-17a6-4b29-9bc2-68e79c353d1c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:27.113102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546210e-ab70-4fad-9809-767ca7986427 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:27.156984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a689198-11d3-4ea8-9e25-90b7ff33bc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:27.158809 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a689198-11d3-4ea8-9e25-90b7ff33bc1c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.159487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a689198-11d3-4ea8-9e25-90b7ff33bc1c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.161309 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a689198-11d3-4ea8-9e25-90b7ff33bc1c None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.162811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a689198-11d3-4ea8-9e25-90b7ff33bc1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:27.163551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a689198-11d3-4ea8-9e25-90b7ff33bc1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:27.165977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a689198-11d3-4ea8-9e25-90b7ff33bc1c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:27.172708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a4bea-2782-4b6b-8131-79cce9451745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:27.177523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a4bea-2782-4b6b-8131-79cce9451745 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.178148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a4bea-2782-4b6b-8131-79cce9451745 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.178615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a4bea-2782-4b6b-8131-79cce9451745 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.180335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a4bea-2782-4b6b-8131-79cce9451745 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:27.184416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a4bea-2782-4b6b-8131-79cce9451745 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:27.186965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 119/229] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:26 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:27.190769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 111/230] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:27.196274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a689198-11d3-4ea8-9e25-90b7ff33bc1c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:27.196450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a4bea-2782-4b6b-8131-79cce9451745 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:27.356809 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:27.367444 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a689198-11d3-4ea8-9e25-90b7ff33bc1c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:27.418245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a4bea-2782-4b6b-8131-79cce9451745 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:27.440387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1724f9-8be3-41a0-ae93-4b6f0ebf2f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:27.441980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1724f9-8be3-41a0-ae93-4b6f0ebf2f34 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.443962 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1724f9-8be3-41a0-ae93-4b6f0ebf2f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.444858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1724f9-8be3-41a0-ae93-4b6f0ebf2f34 None admin] PATH_INFO: `/v3/users/42862c6f25c1440eab3b05c40bdece91` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.447421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1724f9-8be3-41a0-ae93-4b6f0ebf2f34 None admin] RBAC: Authorizing `identity:get_user(user_id=42862c6f25c1440eab3b05c40bdece91)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:27.449097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1724f9-8be3-41a0-ae93-4b6f0ebf2f34 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:27.451338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 112/231] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:27 2022] GET /identity/v3/users/42862c6f25c1440eab3b05c40bdece91 => generated 283 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:27.457951 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1724f9-8be3-41a0-ae93-4b6f0ebf2f34 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:27.500847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59fe34-720f-4f84-a7b8-09bcf93fcaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:27.502128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59fe34-720f-4f84-a7b8-09bcf93fcaab None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.502443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59fe34-720f-4f84-a7b8-09bcf93fcaab None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.502952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59fe34-720f-4f84-a7b8-09bcf93fcaab None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888/users/42862c6f25c1440eab3b05c40bdece91/roles/7562948bd63d483cad68427819d434a9` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.510000 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59fe34-720f-4f84-a7b8-09bcf93fcaab None admin] RBAC: Authorizing `identity:create_grant(project_id=dcbc59eca63d4c3ea02d6e944948c888, user_id=42862c6f25c1440eab3b05c40bdece91, role_id=7562948bd63d483cad68427819d434a9)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:27.511526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59fe34-720f-4f84-a7b8-09bcf93fcaab None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:27.545542 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 120/232] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:02:27 2022] PUT /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888/users/42862c6f25c1440eab3b05c40bdece91/roles/7562948bd63d483cad68427819d434a9 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:27.624535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1724f9-8be3-41a0-ae93-4b6f0ebf2f34 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:27.675036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ee108-d9ea-46be-81ff-2a67d29ac538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:27.676071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ee108-d9ea-46be-81ff-2a67d29ac538 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.680052 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ee108-d9ea-46be-81ff-2a67d29ac538 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.680984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ee108-d9ea-46be-81ff-2a67d29ac538 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.683143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ee108-d9ea-46be-81ff-2a67d29ac538 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:27.685041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ee108-d9ea-46be-81ff-2a67d29ac538 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:27.687069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 113/233] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:27 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:27.695282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59fe34-720f-4f84-a7b8-09bcf93fcaab None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:27.732705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:27.790948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f3b22-6e0b-4aea-a67f-efd6eb37bfd0 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.791265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f3b22-6e0b-4aea-a67f-efd6eb37bfd0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.791543 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f3b22-6e0b-4aea-a67f-efd6eb37bfd0 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.793116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 114/234] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:02:27.808215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dcca1-88f4-4259-bbfe-9369a3dcb7b8 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.808655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dcca1-88f4-4259-bbfe-9369a3dcb7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.809068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dcca1-88f4-4259-bbfe-9369a3dcb7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.836305 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59fe34-720f-4f84-a7b8-09bcf93fcaab None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:27.848391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-842dcca1-88f4-4259-bbfe-9369a3dcb7b8 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:27.849978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-842dcca1-88f4-4259-bbfe-9369a3dcb7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:27.881040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506ddab-6969-427f-8a0d-fae150c6ee5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:27.882444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506ddab-6969-427f-8a0d-fae150c6ee5e None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.882908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506ddab-6969-427f-8a0d-fae150c6ee5e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.883385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506ddab-6969-427f-8a0d-fae150c6ee5e None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888/users/42862c6f25c1440eab3b05c40bdece91/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.886028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506ddab-6969-427f-8a0d-fae150c6ee5e None admin] RBAC: Authorizing `identity:create_grant(project_id=dcbc59eca63d4c3ea02d6e944948c888, user_id=42862c6f25c1440eab3b05c40bdece91, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:27.887940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506ddab-6969-427f-8a0d-fae150c6ee5e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:27.901806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:27.904918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 121/235] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:02:27 2022] PUT /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888/users/42862c6f25c1440eab3b05c40bdece91/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:27.906418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74d04d-3e41-4c01-8ee6-19eb6f011707 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.906702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74d04d-3e41-4c01-8ee6-19eb6f011707 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.906981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74d04d-3e41-4c01-8ee6-19eb6f011707 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.916916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 122/236] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:27.924897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1698b-f437-4ae7-8246-2ecf93d1c307 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.925201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1698b-f437-4ae7-8246-2ecf93d1c307 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.925442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1698b-f437-4ae7-8246-2ecf93d1c307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.981898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-eeb1698b-f437-4ae7-8246-2ecf93d1c307 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:27.983690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-eeb1698b-f437-4ae7-8246-2ecf93d1c307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:28.013901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 115/237] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:28.086944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 123/238] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:28.234143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a6d2d-e912-4eb0-b524-ff423d39edc1 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:28.234432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a6d2d-e912-4eb0-b524-ff423d39edc1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:28.234610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a6d2d-e912-4eb0-b524-ff423d39edc1 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:28.236573 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 116/239] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:28.242808 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed28cd8-6a2b-467a-a6f4-e5ef13a6b8a6 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:28.243399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed28cd8-6a2b-467a-a6f4-e5ef13a6b8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:28.243879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed28cd8-6a2b-467a-a6f4-e5ef13a6b8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:28.269196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-4ed28cd8-6a2b-467a-a6f4-e5ef13a6b8a6 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:28.270722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4ed28cd8-6a2b-467a-a6f4-e5ef13a6b8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:28.296202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:28.355076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a6d2d-e912-4eb0-b524-ff423d39edc1 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:28.392523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 124/240] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:02:28.420852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed28cd8-6a2b-467a-a6f4-e5ef13a6b8a6 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:28.485694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a6d2d-e912-4eb0-b524-ff423d39edc1 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:28.526501 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aaa9f-8661-4821-868a-d3c8a288c885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:28.527146 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aaa9f-8661-4821-868a-d3c8a288c885 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:28.527428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aaa9f-8661-4821-868a-d3c8a288c885 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:28.527700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aaa9f-8661-4821-868a-d3c8a288c885 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:28.528854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aaa9f-8661-4821-868a-d3c8a288c885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:28.530612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aaa9f-8661-4821-868a-d3c8a288c885 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:28.532568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 117/241] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:02:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:28.539051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aaa9f-8661-4821-868a-d3c8a288c885 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:28.594210 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:28.633899 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed28cd8-6a2b-467a-a6f4-e5ef13a6b8a6 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:28.668344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f07c1-3040-4e97-a6d7-0189f6a0ca64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:28.668999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f07c1-3040-4e97-a6d7-0189f6a0ca64 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:28.669334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f07c1-3040-4e97-a6d7-0189f6a0ca64 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:28.669617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f07c1-3040-4e97-a6d7-0189f6a0ca64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:28.693032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f07c1-3040-4e97-a6d7-0189f6a0ca64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-b37f07c1-3040-4e97-a6d7-0189f6a0ca64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:28.706354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:28.707332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:28.708145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 125/242] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:28.711239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aaa9f-8661-4821-868a-d3c8a288c885 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:28.717149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f07c1-3040-4e97-a6d7-0189f6a0ca64 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:28.731260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:28.752799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59a991-4a07-4042-86fe-1d0635db4037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:28.753807 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a991-4a07-4042-86fe-1d0635db4037 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:28.754311 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a991-4a07-4042-86fe-1d0635db4037 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:28.754799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a991-4a07-4042-86fe-1d0635db4037 None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:28.755711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59a991-4a07-4042-86fe-1d0635db4037 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:28.757413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59a991-4a07-4042-86fe-1d0635db4037 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:28.792762 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 118/243] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:02:28 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:28.811737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59a991-4a07-4042-86fe-1d0635db4037 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:28.935827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f07c1-3040-4e97-a6d7-0189f6a0ca64 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:28.975253 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9169f-e7cd-4ee1-9600-e28e141990f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:28.976297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9169f-e7cd-4ee1-9600-e28e141990f5 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:28.976844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9169f-e7cd-4ee1-9600-e28e141990f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:28.977360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9169f-e7cd-4ee1-9600-e28e141990f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:28.978306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9169f-e7cd-4ee1-9600-e28e141990f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:28.978811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9169f-e7cd-4ee1-9600-e28e141990f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:28.980370 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:28.980677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9169f-e7cd-4ee1-9600-e28e141990f5 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:28.991398 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 126/244] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:28.998942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9169f-e7cd-4ee1-9600-e28e141990f5 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:29.001564 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59a991-4a07-4042-86fe-1d0635db4037 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.039374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff3cf5-7ccd-4cb6-a205-17347d30e376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:29.039946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff3cf5-7ccd-4cb6-a205-17347d30e376 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.040289 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff3cf5-7ccd-4cb6-a205-17347d30e376 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.040563 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff3cf5-7ccd-4cb6-a205-17347d30e376 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.051574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff3cf5-7ccd-4cb6-a205-17347d30e376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-8cff3cf5-7ccd-4cb6-a205-17347d30e376 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:29.056911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:29.057915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:29.058829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 119/245] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:02:29.066589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff3cf5-7ccd-4cb6-a205-17347d30e376 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:29.097199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:29.141477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9169f-e7cd-4ee1-9600-e28e141990f5 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.180009 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7128185-942b-47cd-89e5-33abb298c096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:29.180863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7128185-942b-47cd-89e5-33abb298c096 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.181256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7128185-942b-47cd-89e5-33abb298c096 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.181654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7128185-942b-47cd-89e5-33abb298c096 None admin] PATH_INFO: `/v3/users/32ff4ee17e2f45fc96ed4562ec48e88b` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.193103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff3cf5-7ccd-4cb6-a205-17347d30e376 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.198635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7128185-942b-47cd-89e5-33abb298c096 None admin] RBAC: Authorizing `identity:get_user(user_id=32ff4ee17e2f45fc96ed4562ec48e88b)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:29.200354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7128185-942b-47cd-89e5-33abb298c096 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:29.202341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 127/246] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:28 2022] GET /identity/v3/users/32ff4ee17e2f45fc96ed4562ec48e88b => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:29.208726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7128185-942b-47cd-89e5-33abb298c096 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:29.238274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b9ab1-938a-43e4-9ae7-5301dc41a462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:29.239205 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b9ab1-938a-43e4-9ae7-5301dc41a462 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.239616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b9ab1-938a-43e4-9ae7-5301dc41a462 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.240048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b9ab1-938a-43e4-9ae7-5301dc41a462 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.240997 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b9ab1-938a-43e4-9ae7-5301dc41a462 None admin] RBAC: Adding query filter params (name=member) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:29.241510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b9ab1-938a-43e4-9ae7-5301dc41a462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:29.243092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b9ab1-938a-43e4-9ae7-5301dc41a462 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:29.252593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 120/247] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:29.258160 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b9ab1-938a-43e4-9ae7-5301dc41a462 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:29.263170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:29.324452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7128185-942b-47cd-89e5-33abb298c096 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.355157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:29.364900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212e5ac-d951-4f39-a37a-090be0dae809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:29.365683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212e5ac-d951-4f39-a37a-090be0dae809 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.366060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212e5ac-d951-4f39-a37a-090be0dae809 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.366447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212e5ac-d951-4f39-a37a-090be0dae809 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.380754 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b9ab1-938a-43e4-9ae7-5301dc41a462 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.419889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5604217-6ca0-4a6e-ba65-807f936df8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:29.421115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5604217-6ca0-4a6e-ba65-807f936df8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.421517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5604217-6ca0-4a6e-ba65-807f936df8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.422034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5604217-6ca0-4a6e-ba65-807f936df8a1 None admin] PATH_INFO: `/v3/users/f0194d28c9454e32b4d3a9ea7f5e7178` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.424117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5604217-6ca0-4a6e-ba65-807f936df8a1 None admin] RBAC: Authorizing `identity:get_user(user_id=f0194d28c9454e32b4d3a9ea7f5e7178)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:29.425869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5604217-6ca0-4a6e-ba65-807f936df8a1 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:29.428364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 121/248] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:29 2022] GET /identity/v3/users/f0194d28c9454e32b4d3a9ea7f5e7178 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:29.434517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5604217-6ca0-4a6e-ba65-807f936df8a1 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:29.449026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212e5ac-d951-4f39-a37a-090be0dae809 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:29.450810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212e5ac-d951-4f39-a37a-090be0dae809 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:29.518505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:29.554528 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5604217-6ca0-4a6e-ba65-807f936df8a1 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.567416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 128/249] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:29.574169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212e5ac-d951-4f39-a37a-090be0dae809 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:29.594261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d1367-0097-4715-8196-ae714eba5e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:29.594823 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d1367-0097-4715-8196-ae714eba5e6e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.595094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d1367-0097-4715-8196-ae714eba5e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.595901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d1367-0097-4715-8196-ae714eba5e6e None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.597262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d1367-0097-4715-8196-ae714eba5e6e None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:29.598883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d1367-0097-4715-8196-ae714eba5e6e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:29.601024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 122/250] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:29 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:29.607922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d1367-0097-4715-8196-ae714eba5e6e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:29.612315 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:29.705911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212e5ac-d951-4f39-a37a-090be0dae809 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.747196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d1367-0097-4715-8196-ae714eba5e6e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.754179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d6901-1c19-4089-a154-fc625a626512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:29.755115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d6901-1c19-4089-a154-fc625a626512 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.755666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d6901-1c19-4089-a154-fc625a626512 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.756178 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d6901-1c19-4089-a154-fc625a626512 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.757626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d6901-1c19-4089-a154-fc625a626512 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:29.759397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d6901-1c19-4089-a154-fc625a626512 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:29.761697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 129/251] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:29 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:29.767846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d6901-1c19-4089-a154-fc625a626512 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:29.785171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa53269-049f-4a81-877a-8dcb124a6e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:29.785809 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa53269-049f-4a81-877a-8dcb124a6e85 None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.786086 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa53269-049f-4a81-877a-8dcb124a6e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.786358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa53269-049f-4a81-877a-8dcb124a6e85 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888/users/f0194d28c9454e32b4d3a9ea7f5e7178/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.792037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa53269-049f-4a81-877a-8dcb124a6e85 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcbc59eca63d4c3ea02d6e944948c888, user_id=f0194d28c9454e32b4d3a9ea7f5e7178, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:29.792037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa53269-049f-4a81-877a-8dcb124a6e85 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:29.803316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:29.813228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 123/252] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:02:29 2022] PUT /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888/users/f0194d28c9454e32b4d3a9ea7f5e7178/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:29.947129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:29.953837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d6901-1c19-4089-a154-fc625a626512 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:29.968778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c39cef-03a6-4e9c-8799-cdf92575687d None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.969112 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c39cef-03a6-4e9c-8799-cdf92575687d None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.969329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c39cef-03a6-4e9c-8799-cdf92575687d None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:29.973932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 124/253] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:02:29.988498 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df481d-1016-41e3-a4e0-df562cbc264a None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:29.988821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df481d-1016-41e3-a4e0-df562cbc264a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:29.989036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df481d-1016-41e3-a4e0-df562cbc264a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.024913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-67df481d-1016-41e3-a4e0-df562cbc264a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:30.026765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-67df481d-1016-41e3-a4e0-df562cbc264a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:30.035495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103a87d-020f-46b1-998a-e69b7bf8d622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:30.036272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103a87d-020f-46b1-998a-e69b7bf8d622 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.036549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103a87d-020f-46b1-998a-e69b7bf8d622 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.036861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103a87d-020f-46b1-998a-e69b7bf8d622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.037725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103a87d-020f-46b1-998a-e69b7bf8d622 None admin] RBAC: Adding query filter params (role.id=cd6b754c94e54de6b3bc5e17618f1e67, scope.project.id=dcbc59eca63d4c3ea02d6e944948c888, user.id=32ff4ee17e2f45fc96ed4562ec48e88b) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:30.038082 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103a87d-020f-46b1-998a-e69b7bf8d622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:30.039557 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103a87d-020f-46b1-998a-e69b7bf8d622 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:30.056681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 130/254] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:29 2022] GET /identity/v3/role_assignments?user.id=32ff4ee17e2f45fc96ed4562ec48e88b&scope.project.id=dcbc59eca63d4c3ea02d6e944948c888&role.id=cd6b754c94e54de6b3bc5e17618f1e67 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:30.139301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 125/255] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:02:30.503321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01baec-e7f6-4e8e-a7e4-2d19c7557a41 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.503695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01baec-e7f6-4e8e-a7e4-2d19c7557a41 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.503891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01baec-e7f6-4e8e-a7e4-2d19c7557a41 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.505404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 131/256] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:30.510737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0195ec-9a81-434c-8f0a-10e4a7fbdd24 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.511083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0195ec-9a81-434c-8f0a-10e4a7fbdd24 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.511359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0195ec-9a81-434c-8f0a-10e4a7fbdd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.535198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-1c0195ec-9a81-434c-8f0a-10e4a7fbdd24 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:30.536399 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1c0195ec-9a81-434c-8f0a-10e4a7fbdd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:30.558326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01baec-e7f6-4e8e-a7e4-2d19c7557a41 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:30.639581 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 126/257] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:30.772347 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01baec-e7f6-4e8e-a7e4-2d19c7557a41 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:30.818132 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6f468-f3c2-470e-967a-298931d2fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:30.818784 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6f468-f3c2-470e-967a-298931d2fd5a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.819063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6f468-f3c2-470e-967a-298931d2fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.819337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6f468-f3c2-470e-967a-298931d2fd5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.839157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6f468-f3c2-470e-967a-298931d2fd5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-14f6f468-f3c2-470e-967a-298931d2fd5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:30.844533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:30.845440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:30.846251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 132/258] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:02:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:30.851721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0195ec-9a81-434c-8f0a-10e4a7fbdd24 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:30.931287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:30.983923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0195ec-9a81-434c-8f0a-10e4a7fbdd24 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.047047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73f8d-709d-4d2a-bdb2-6a782a38dc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.047866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73f8d-709d-4d2a-bdb2-6a782a38dc12 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.048377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73f8d-709d-4d2a-bdb2-6a782a38dc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.049275 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73f8d-709d-4d2a-bdb2-6a782a38dc12 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.050319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f73f8d-709d-4d2a-bdb2-6a782a38dc12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:31.050785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f73f8d-709d-4d2a-bdb2-6a782a38dc12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.053009 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f73f8d-709d-4d2a-bdb2-6a782a38dc12 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.070005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 127/259] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:02:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:31.077096 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6f468-f3c2-470e-967a-298931d2fd5a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.114749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:31.138800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73f8d-709d-4d2a-bdb2-6a782a38dc12 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.263038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6f468-f3c2-470e-967a-298931d2fd5a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.349330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aafc9-4527-4b52-85f0-1b2d60fb4725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.350973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aafc9-4527-4b52-85f0-1b2d60fb4725 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.351601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aafc9-4527-4b52-85f0-1b2d60fb4725 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.353138 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aafc9-4527-4b52-85f0-1b2d60fb4725 None admin] PATH_INFO: `/v3/users/42862c6f25c1440eab3b05c40bdece91` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.360023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aafc9-4527-4b52-85f0-1b2d60fb4725 None admin] RBAC: Authorizing `identity:get_user(user_id=42862c6f25c1440eab3b05c40bdece91)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.362117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aafc9-4527-4b52-85f0-1b2d60fb4725 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.366792 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73f8d-709d-4d2a-bdb2-6a782a38dc12 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.368677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 133/260] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:31 2022] GET /identity/v3/users/42862c6f25c1440eab3b05c40bdece91 => generated 283 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:31.373289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aafc9-4527-4b52-85f0-1b2d60fb4725 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.439037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:31.467114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc53a8-efe7-46e8-877d-700facff728a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.468343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc53a8-efe7-46e8-877d-700facff728a None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.469451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc53a8-efe7-46e8-877d-700facff728a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.470079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc53a8-efe7-46e8-877d-700facff728a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.496187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc53a8-efe7-46e8-877d-700facff728a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-a0fc53a8-efe7-46e8-877d-700facff728a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:31.503254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:31.504161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:31.504987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:31.504987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:31.504987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:31.504987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:31.504987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:31.504987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:31.504987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:31.504987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:31.504987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:31.504987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:31.504987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:31.504987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:31.513922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 128/261] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:02:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:02:31.519810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc53a8-efe7-46e8-877d-700facff728a None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.599696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:31.624558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aafc9-4527-4b52-85f0-1b2d60fb4725 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.699955 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc53a8-efe7-46e8-877d-700facff728a None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.711977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce224a2-f8d0-48e4-b9db-af0b0586fe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.712563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce224a2-f8d0-48e4-b9db-af0b0586fe1b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.712833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce224a2-f8d0-48e4-b9db-af0b0586fe1b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.713104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce224a2-f8d0-48e4-b9db-af0b0586fe1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.749694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b72b7-cb6b-4c2d-90f7-9276ccfc15f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.751210 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b72b7-cb6b-4c2d-90f7-9276ccfc15f1 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.754473 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b72b7-cb6b-4c2d-90f7-9276ccfc15f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.755007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b72b7-cb6b-4c2d-90f7-9276ccfc15f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.755893 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b72b7-cb6b-4c2d-90f7-9276ccfc15f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:31.756414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b72b7-cb6b-4c2d-90f7-9276ccfc15f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.757983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b72b7-cb6b-4c2d-90f7-9276ccfc15f1 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.780421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 129/262] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:31.805485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b72b7-cb6b-4c2d-90f7-9276ccfc15f1 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.839759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce224a2-f8d0-48e4-b9db-af0b0586fe1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.841456 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce224a2-f8d0-48e4-b9db-af0b0586fe1b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.912973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:31.932442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:31.952052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 134/263] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 01:02:31.952052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce224a2-f8d0-48e4-b9db-af0b0586fe1b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:32.037987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b72b7-cb6b-4c2d-90f7-9276ccfc15f1 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:32.056501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce224a2-f8d0-48e4-b9db-af0b0586fe1b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:32.092552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d7fcd-da5c-44fc-b0a5-5d3b1910f530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:32.093187 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d7fcd-da5c-44fc-b0a5-5d3b1910f530 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.093464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d7fcd-da5c-44fc-b0a5-5d3b1910f530 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.093730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d7fcd-da5c-44fc-b0a5-5d3b1910f530 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.094896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d7fcd-da5c-44fc-b0a5-5d3b1910f530 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:32.096967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d7fcd-da5c-44fc-b0a5-5d3b1910f530 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:32.103821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 135/264] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:31 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:32.105208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d7fcd-da5c-44fc-b0a5-5d3b1910f530 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:32.164081 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b9980-3eba-49b4-a005-b02e3a6dfccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:32.164653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b9980-3eba-49b4-a005-b02e3a6dfccb None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.164936 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b9980-3eba-49b4-a005-b02e3a6dfccb None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.165212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b9980-3eba-49b4-a005-b02e3a6dfccb None admin] PATH_INFO: `/v3/users/42862c6f25c1440eab3b05c40bdece91` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.166381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b9980-3eba-49b4-a005-b02e3a6dfccb None admin] RBAC: Authorizing `identity:get_user(user_id=42862c6f25c1440eab3b05c40bdece91)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:32.167859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b9980-3eba-49b4-a005-b02e3a6dfccb None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:32.169832 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 130/265] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:31 2022] GET /identity/v3/users/42862c6f25c1440eab3b05c40bdece91 => generated 283 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:32.197075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b9980-3eba-49b4-a005-b02e3a6dfccb None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:32.216726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:32.219307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d7fcd-da5c-44fc-b0a5-5d3b1910f530 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:32.257866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d982ef4-21db-4b69-af1d-96c8574c6b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:32.258531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d982ef4-21db-4b69-af1d-96c8574c6b3b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.258804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d982ef4-21db-4b69-af1d-96c8574c6b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.259212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d982ef4-21db-4b69-af1d-96c8574c6b3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.259943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d982ef4-21db-4b69-af1d-96c8574c6b3b None admin] RBAC: Adding query filter params (role.id=7562948bd63d483cad68427819d434a9, scope.project.id=dcbc59eca63d4c3ea02d6e944948c888, user.id=42862c6f25c1440eab3b05c40bdece91) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:32.260276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d982ef4-21db-4b69-af1d-96c8574c6b3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:32.261731 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d982ef4-21db-4b69-af1d-96c8574c6b3b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:32.269302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 136/266] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:32 2022] GET /identity/v3/role_assignments?user.id=42862c6f25c1440eab3b05c40bdece91&scope.project.id=dcbc59eca63d4c3ea02d6e944948c888&role.id=7562948bd63d483cad68427819d434a9 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:32.365791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b9980-3eba-49b4-a005-b02e3a6dfccb None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:32.387285 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:32.419141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8afe6c-fe2c-4f13-a5f2-6c2a187c710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:32.419686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8afe6c-fe2c-4f13-a5f2-6c2a187c710a None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.419967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8afe6c-fe2c-4f13-a5f2-6c2a187c710a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.420522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8afe6c-fe2c-4f13-a5f2-6c2a187c710a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.520366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8afe6c-fe2c-4f13-a5f2-6c2a187c710a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:32.521872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8afe6c-fe2c-4f13-a5f2-6c2a187c710a None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:32.528911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a58eb-febe-4919-99f0-fe20d40939bf None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.529216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a58eb-febe-4919-99f0-fe20d40939bf None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.529461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a58eb-febe-4919-99f0-fe20d40939bf None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.535625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 137/267] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:32.541605 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb40a82-0eb1-4b97-a88f-6e43aa6461f6 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.541896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb40a82-0eb1-4b97-a88f-6e43aa6461f6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.542140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb40a82-0eb1-4b97-a88f-6e43aa6461f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.593248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-bfb40a82-0eb1-4b97-a88f-6e43aa6461f6 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:32.594556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-bfb40a82-0eb1-4b97-a88f-6e43aa6461f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:32.709590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 138/268] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:02:32.725553 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 131/269] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:32.731365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb40a82-0eb1-4b97-a88f-6e43aa6461f6 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:32.786514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01596bef-b8ab-4960-a5cd-842215f000c8 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.786819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01596bef-b8ab-4960-a5cd-842215f000c8 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.787062 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01596bef-b8ab-4960-a5cd-842215f000c8 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.788742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 132/270] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:32.791019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:32.797004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5117d-646d-4164-88e2-dcdb436f2290 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.797422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5117d-646d-4164-88e2-dcdb436f2290 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.797798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5117d-646d-4164-88e2-dcdb436f2290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.825984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-95a5117d-646d-4164-88e2-dcdb436f2290 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:32.827358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-95a5117d-646d-4164-88e2-dcdb436f2290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:32.844026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:32.883756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb40a82-0eb1-4b97-a88f-6e43aa6461f6 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:32.913801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 133/271] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:32.914896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0267c-93a2-426b-8370-7f66ffe6132e None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.915184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0267c-93a2-426b-8370-7f66ffe6132e None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.915459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0267c-93a2-426b-8370-7f66ffe6132e None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.916940 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 134/272] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:32.917898 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a40fc-23fd-4823-b6d3-56cfe83b72ee None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.918078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a40fc-23fd-4823-b6d3-56cfe83b72ee None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.918346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a40fc-23fd-4823-b6d3-56cfe83b72ee None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.919725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 135/273] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:32.922264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482f8eb-e708-4544-8773-0d9154f0028b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.922550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482f8eb-e708-4544-8773-0d9154f0028b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.922849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482f8eb-e708-4544-8773-0d9154f0028b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.935098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60cdc5-ab11-462e-b4b9-f73d2f601fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:32.935776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60cdc5-ab11-462e-b4b9-f73d2f601fee None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.936189 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60cdc5-ab11-462e-b4b9-f73d2f601fee None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.936467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60cdc5-ab11-462e-b4b9-f73d2f601fee None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.939948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60cdc5-ab11-462e-b4b9-f73d2f601fee None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:32.941583 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60cdc5-ab11-462e-b4b9-f73d2f601fee None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:32.942285 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-0482f8eb-e708-4544-8773-0d9154f0028b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:32.943462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0482f8eb-e708-4544-8773-0d9154f0028b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:32.943696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 139/274] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:32 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:32.944841 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232f743-465e-4ade-9df7-2f57ab3ba50e None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:32.945100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232f743-465e-4ade-9df7-2f57ab3ba50e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:32.945373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232f743-465e-4ade-9df7-2f57ab3ba50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:32.965437 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7232f743-465e-4ade-9df7-2f57ab3ba50e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:32.966603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7232f743-465e-4ade-9df7-2f57ab3ba50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:33.017637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 136/275] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:33.018811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482f8eb-e708-4544-8773-0d9154f0028b None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.059754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 140/276] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:33.101496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232f743-465e-4ade-9df7-2f57ab3ba50e None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.132279 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:33.168878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482f8eb-e708-4544-8773-0d9154f0028b None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.192477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:33.210896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748536b3-b924-46f9-b6c7-063e9c7b3208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.211544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748536b3-b924-46f9-b6c7-063e9c7b3208 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.211824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748536b3-b924-46f9-b6c7-063e9c7b3208 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.212131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748536b3-b924-46f9-b6c7-063e9c7b3208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.212885 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748536b3-b924-46f9-b6c7-063e9c7b3208 None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=dcbc59eca63d4c3ea02d6e944948c888, user.id=42862c6f25c1440eab3b05c40bdece91) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:33.213194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748536b3-b924-46f9-b6c7-063e9c7b3208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.214740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748536b3-b924-46f9-b6c7-063e9c7b3208 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.221892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 137/277] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/role_assignments?user.id=42862c6f25c1440eab3b05c40bdece91&scope.project.id=dcbc59eca63d4c3ea02d6e944948c888&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:33.222681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748536b3-b924-46f9-b6c7-063e9c7b3208 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.229219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232f743-465e-4ade-9df7-2f57ab3ba50e None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.284733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d574f85-4f4d-49a7-a212-afbbab40d5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.285390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d574f85-4f4d-49a7-a212-afbbab40d5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.285665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d574f85-4f4d-49a7-a212-afbbab40d5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.285933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d574f85-4f4d-49a7-a212-afbbab40d5d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.304635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d574f85-4f4d-49a7-a212-afbbab40d5d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-5d574f85-4f4d-49a7-a212-afbbab40d5d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:33.310131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:33.311170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:33.312676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 141/278] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:02:33.318255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d574f85-4f4d-49a7-a212-afbbab40d5d8 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.391100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748536b3-b924-46f9-b6c7-063e9c7b3208 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.470724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b433e-0bb4-4ce5-abba-ca52a1dcf181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.471468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b433e-0bb4-4ce5-abba-ca52a1dcf181 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.471979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b433e-0bb4-4ce5-abba-ca52a1dcf181 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.472425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b433e-0bb4-4ce5-abba-ca52a1dcf181 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.480267 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:33.498949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b433e-0bb4-4ce5-abba-ca52a1dcf181 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-ea4b433e-0bb4-4ce5-abba-ca52a1dcf181 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:33.511312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:33.512720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:33.513829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:33.513829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:33.513829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:33.513829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:33.513829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:33.513829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:33.513829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:33.513829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:33.513829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:33.513829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:33.513829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:33.513829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:33.514419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d574f85-4f4d-49a7-a212-afbbab40d5d8 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.514503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 138/279] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:33.514503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b433e-0bb4-4ce5-abba-ca52a1dcf181 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.522129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:33.549861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1a0d5-a681-45b5-8772-69fb87c9f41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.550820 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1a0d5-a681-45b5-8772-69fb87c9f41b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.551244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1a0d5-a681-45b5-8772-69fb87c9f41b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.551511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1a0d5-a681-45b5-8772-69fb87c9f41b None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.552244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1a0d5-a681-45b5-8772-69fb87c9f41b None admin] RBAC: Adding query filter params (name=member) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:33.552568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1a0d5-a681-45b5-8772-69fb87c9f41b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.553921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1a0d5-a681-45b5-8772-69fb87c9f41b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.562429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 142/280] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:33.564647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1a0d5-a681-45b5-8772-69fb87c9f41b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.633570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b433e-0bb4-4ce5-abba-ca52a1dcf181 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.668754 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc908b24-ee06-46b3-9465-22738620a61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.669391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc908b24-ee06-46b3-9465-22738620a61f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.669672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc908b24-ee06-46b3-9465-22738620a61f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.669942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc908b24-ee06-46b3-9465-22738620a61f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.682545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc908b24-ee06-46b3-9465-22738620a61f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:33.687045 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1a0d5-a681-45b5-8772-69fb87c9f41b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-cc908b24-ee06-46b3-9465-22738620a61f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:33.687942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:33.689585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 139/281] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:33.690993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc908b24-ee06-46b3-9465-22738620a61f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.729666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080cb70-9ab8-48dc-9119-f8f63a6f601b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.730365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080cb70-9ab8-48dc-9119-f8f63a6f601b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.730738 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080cb70-9ab8-48dc-9119-f8f63a6f601b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.731120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080cb70-9ab8-48dc-9119-f8f63a6f601b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.732480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080cb70-9ab8-48dc-9119-f8f63a6f601b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.734494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080cb70-9ab8-48dc-9119-f8f63a6f601b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.736234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 143/282] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:33.737541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080cb70-9ab8-48dc-9119-f8f63a6f601b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.759898 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:33.784760 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:33.812886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc908b24-ee06-46b3-9465-22738620a61f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.852960 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24c65a-7ccd-4f8d-a892-40d7491b4ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.853346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24c65a-7ccd-4f8d-a892-40d7491b4ebb None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.853617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24c65a-7ccd-4f8d-a892-40d7491b4ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.853831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24c65a-7ccd-4f8d-a892-40d7491b4ebb None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.854509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24c65a-7ccd-4f8d-a892-40d7491b4ebb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:33.854804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24c65a-7ccd-4f8d-a892-40d7491b4ebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.856215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24c65a-7ccd-4f8d-a892-40d7491b4ebb None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.860061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080cb70-9ab8-48dc-9119-f8f63a6f601b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.867066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 140/283] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:33.869393 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24c65a-7ccd-4f8d-a892-40d7491b4ebb None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.900867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f45e98-fea3-4346-9a7c-9ad6f1555aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:33.901984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f45e98-fea3-4346-9a7c-9ad6f1555aba None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:33.902402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f45e98-fea3-4346-9a7c-9ad6f1555aba None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:33.902834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f45e98-fea3-4346-9a7c-9ad6f1555aba None admin] PATH_INFO: `/v3/users/4c1073563c9f48eb9c8458e6f35aa456` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:33.918809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f45e98-fea3-4346-9a7c-9ad6f1555aba None admin] RBAC: Authorizing `identity:get_user(user_id=4c1073563c9f48eb9c8458e6f35aa456)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:33.920608 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f45e98-fea3-4346-9a7c-9ad6f1555aba None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:33.922299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 144/284] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/users/4c1073563c9f48eb9c8458e6f35aa456 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:33.923052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f45e98-fea3-4346-9a7c-9ad6f1555aba None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:33.981268 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24c65a-7ccd-4f8d-a892-40d7491b4ebb None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:33.998346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:34.016332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e71f8-bb8c-44d1-b065-2baaaf7ee29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.017906 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e71f8-bb8c-44d1-b065-2baaaf7ee29e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.018328 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e71f8-bb8c-44d1-b065-2baaaf7ee29e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.018604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e71f8-bb8c-44d1-b065-2baaaf7ee29e None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.020214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e71f8-bb8c-44d1-b065-2baaaf7ee29e None admin] RBAC: Adding query filter params (name=member) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:34.020551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e71f8-bb8c-44d1-b065-2baaaf7ee29e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.021912 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e71f8-bb8c-44d1-b065-2baaaf7ee29e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.031025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 141/285] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:34.031852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e71f8-bb8c-44d1-b065-2baaaf7ee29e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.037488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f45e98-fea3-4346-9a7c-9ad6f1555aba None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.039232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:34.078791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9000104-bd72-4d2b-97a0-22a9d6083f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.079403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9000104-bd72-4d2b-97a0-22a9d6083f35 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.080959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9000104-bd72-4d2b-97a0-22a9d6083f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.081245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9000104-bd72-4d2b-97a0-22a9d6083f35 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.081913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9000104-bd72-4d2b-97a0-22a9d6083f35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.083356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9000104-bd72-4d2b-97a0-22a9d6083f35 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.106285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 145/286] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:02:33 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:34.107190 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9000104-bd72-4d2b-97a0-22a9d6083f35 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.153219 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e71f8-bb8c-44d1-b065-2baaaf7ee29e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.195536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d26b54-cd28-4f53-a3e7-778aa5ee4852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.196297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d26b54-cd28-4f53-a3e7-778aa5ee4852 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.196579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d26b54-cd28-4f53-a3e7-778aa5ee4852 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.196836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d26b54-cd28-4f53-a3e7-778aa5ee4852 None admin] PATH_INFO: `/v3/users/32ff4ee17e2f45fc96ed4562ec48e88b` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.198131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d26b54-cd28-4f53-a3e7-778aa5ee4852 None admin] RBAC: Authorizing `identity:get_user(user_id=32ff4ee17e2f45fc96ed4562ec48e88b)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.199624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d26b54-cd28-4f53-a3e7-778aa5ee4852 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.201498 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 142/287] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/users/32ff4ee17e2f45fc96ed4562ec48e88b => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:34.202133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d26b54-cd28-4f53-a3e7-778aa5ee4852 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.257602 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9000104-bd72-4d2b-97a0-22a9d6083f35 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.264064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:34.304144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b6937-02a6-4f27-b655-a72bf16edba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.304906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b6937-02a6-4f27-b655-a72bf16edba1 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.305696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b6937-02a6-4f27-b655-a72bf16edba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.306804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b6937-02a6-4f27-b655-a72bf16edba1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.306804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b6937-02a6-4f27-b655-a72bf16edba1 None admin] RBAC: Adding query filter params (role.id=b07ea738cc204b2b8125bf4cd6d9c54a, user.id=4c1073563c9f48eb9c8458e6f35aa456, scope.system=all) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:34.307053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b6937-02a6-4f27-b655-a72bf16edba1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.309258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b6937-02a6-4f27-b655-a72bf16edba1 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.319066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 146/288] 10.222.0.44 () {56 vars in 1308 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/role_assignments?user.id=4c1073563c9f48eb9c8458e6f35aa456&scope.system=all&role.id=b07ea738cc204b2b8125bf4cd6d9c54a => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:34.319066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b6937-02a6-4f27-b655-a72bf16edba1 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.345475 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:34.346833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d26b54-cd28-4f53-a3e7-778aa5ee4852 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.401170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8e643-66fd-4960-b05f-c8c87f29fc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.402006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8e643-66fd-4960-b05f-c8c87f29fc9c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.402473 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8e643-66fd-4960-b05f-c8c87f29fc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.402936 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8e643-66fd-4960-b05f-c8c87f29fc9c None admin] PATH_INFO: `/v3/users/f0194d28c9454e32b4d3a9ea7f5e7178` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.406065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8e643-66fd-4960-b05f-c8c87f29fc9c None admin] RBAC: Authorizing `identity:get_user(user_id=f0194d28c9454e32b4d3a9ea7f5e7178)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.408912 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8e643-66fd-4960-b05f-c8c87f29fc9c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.412271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 143/289] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/users/f0194d28c9454e32b4d3a9ea7f5e7178 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:34.421745 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8e643-66fd-4960-b05f-c8c87f29fc9c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.479824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b6937-02a6-4f27-b655-a72bf16edba1 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.524629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd22bf-316b-4017-a3a4-af4a97652337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.525593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd22bf-316b-4017-a3a4-af4a97652337 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.526066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd22bf-316b-4017-a3a4-af4a97652337 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.526544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd22bf-316b-4017-a3a4-af4a97652337 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.527925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd22bf-316b-4017-a3a4-af4a97652337 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.529645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd22bf-316b-4017-a3a4-af4a97652337 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.531867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 147/290] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:34.543656 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd22bf-316b-4017-a3a4-af4a97652337 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:34.590559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8e643-66fd-4960-b05f-c8c87f29fc9c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.617948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:34.654313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1e1d2-62c2-44bc-9fa7-5e6e0234d0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.656656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1e1d2-62c2-44bc-9fa7-5e6e0234d0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.657370 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1e1d2-62c2-44bc-9fa7-5e6e0234d0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.659326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1e1d2-62c2-44bc-9fa7-5e6e0234d0b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.745330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:34.776356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd22bf-316b-4017-a3a4-af4a97652337 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:34.811061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1e1d2-62c2-44bc-9fa7-5e6e0234d0b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.812559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1e1d2-62c2-44bc-9fa7-5e6e0234d0b2 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.837385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d4617-8e87-48f1-878a-b71e988c2cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:34.838057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d4617-8e87-48f1-878a-b71e988c2cad None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.838330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d4617-8e87-48f1-878a-b71e988c2cad None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.838599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d4617-8e87-48f1-878a-b71e988c2cad None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888/users/32ff4ee17e2f45fc96ed4562ec48e88b/roles/cd6b754c94e54de6b3bc5e17618f1e67` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.842703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d4617-8e87-48f1-878a-b71e988c2cad None admin] RBAC: Authorizing `identity:create_grant(project_id=dcbc59eca63d4c3ea02d6e944948c888, user_id=32ff4ee17e2f45fc96ed4562ec48e88b, role_id=cd6b754c94e54de6b3bc5e17618f1e67)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:34.846294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d4617-8e87-48f1-878a-b71e988c2cad None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:34.870959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 148/291] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:02:34 2022] PUT /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888/users/32ff4ee17e2f45fc96ed4562ec48e88b/roles/cd6b754c94e54de6b3bc5e17618f1e67 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:34.972923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 144/292] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:34.981358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d4617-8e87-48f1-878a-b71e988c2cad None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.130565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d4617-8e87-48f1-878a-b71e988c2cad None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.166809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2f459-be09-488f-ac28-186e98c6437a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.167452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2f459-be09-488f-ac28-186e98c6437a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.167726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2f459-be09-488f-ac28-186e98c6437a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.168035 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2f459-be09-488f-ac28-186e98c6437a None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.169164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2f459-be09-488f-ac28-186e98c6437a None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.170930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2f459-be09-488f-ac28-186e98c6437a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.172922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 149/293] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:34 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:35.191520 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1e1d2-62c2-44bc-9fa7-5e6e0234d0b2 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.219977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:35.326310 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1e1d2-62c2-44bc-9fa7-5e6e0234d0b2 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.364457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795f85c-7f5e-4485-a751-7eb511a13aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.365065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795f85c-7f5e-4485-a751-7eb511a13aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.365336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795f85c-7f5e-4485-a751-7eb511a13aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.365604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795f85c-7f5e-4485-a751-7eb511a13aa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.366330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795f85c-7f5e-4485-a751-7eb511a13aa9 None admin] RBAC: Adding query filter params (role.id=b07ea738cc204b2b8125bf4cd6d9c54a, scope.project.id=dcbc59eca63d4c3ea02d6e944948c888, user.id=f0194d28c9454e32b4d3a9ea7f5e7178) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:35.366632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795f85c-7f5e-4485-a751-7eb511a13aa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.368067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795f85c-7f5e-4485-a751-7eb511a13aa9 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.374529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 145/294] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:35 2022] GET /identity/v3/role_assignments?user.id=f0194d28c9454e32b4d3a9ea7f5e7178&scope.project.id=dcbc59eca63d4c3ea02d6e944948c888&role.id=b07ea738cc204b2b8125bf4cd6d9c54a => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:36.536446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3fca6-370b-447f-828e-df5d4c7726da None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.536995 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3fca6-370b-447f-828e-df5d4c7726da None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.536995 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3fca6-370b-447f-828e-df5d4c7726da None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.538337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 150/295] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:36.543690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d924c-bded-470a-b34e-f6abf48276dc None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.544143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d924c-bded-470a-b34e-f6abf48276dc None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.544238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d924c-bded-470a-b34e-f6abf48276dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.563260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-171d924c-bded-470a-b34e-f6abf48276dc None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:36.564714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-171d924c-bded-470a-b34e-f6abf48276dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:36.634309 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 146/296] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:36.638477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035111a-c944-4906-ae0d-8c2bfd40f67f None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.638760 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035111a-c944-4906-ae0d-8c2bfd40f67f None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.639033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035111a-c944-4906-ae0d-8c2bfd40f67f None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.640399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 151/297] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:36.646386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f152f76-767f-4d08-ae71-d6bd2938f5b3 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.646671 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f152f76-767f-4d08-ae71-d6bd2938f5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.646944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f152f76-767f-4d08-ae71-d6bd2938f5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.667195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-2f152f76-767f-4d08-ae71-d6bd2938f5b3 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:36.668442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2f152f76-767f-4d08-ae71-d6bd2938f5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:36.672137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:36.752363 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 147/298] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:36.773326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d0053-7fa3-4339-90e1-66861d257822 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.773623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d0053-7fa3-4339-90e1-66861d257822 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.773880 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d0053-7fa3-4339-90e1-66861d257822 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.775505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 152/299] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:36.780800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728be28-3376-4d4a-b7a7-4492a6b34a70 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:36.781083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728be28-3376-4d4a-b7a7-4492a6b34a70 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:36.781356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728be28-3376-4d4a-b7a7-4492a6b34a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:36.801007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-0728be28-3376-4d4a-b7a7-4492a6b34a70 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:36.802107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0728be28-3376-4d4a-b7a7-4492a6b34a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:36.872879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 148/300] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:36.922818 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d0053-7fa3-4339-90e1-66861d257822 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:36.939100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:37.035854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d0053-7fa3-4339-90e1-66861d257822 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.056609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728be28-3376-4d4a-b7a7-4492a6b34a70 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.075732 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33a80a-fb3c-4318-b42b-1b54da30cc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.076672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33a80a-fb3c-4318-b42b-1b54da30cc6a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.076946 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33a80a-fb3c-4318-b42b-1b54da30cc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.077215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33a80a-fb3c-4318-b42b-1b54da30cc6a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.090938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33a80a-fb3c-4318-b42b-1b54da30cc6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-ae33a80a-fb3c-4318-b42b-1b54da30cc6a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:37.100145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:37.101221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:37.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 153/301] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:37.109283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33a80a-fb3c-4318-b42b-1b54da30cc6a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.138000 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:37.177079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728be28-3376-4d4a-b7a7-4492a6b34a70 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.200253 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:37.219150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019a5ec-30c9-4246-a450-d58e64703904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.219787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019a5ec-30c9-4246-a450-d58e64703904 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.220672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019a5ec-30c9-4246-a450-d58e64703904 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.221043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019a5ec-30c9-4246-a450-d58e64703904 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.231218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019a5ec-30c9-4246-a450-d58e64703904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:37.232732 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33a80a-fb3c-4318-b42b-1b54da30cc6a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-d019a5ec-30c9-4246-a450-d58e64703904 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:37.236068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:37.237144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 149/302] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:37.237924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019a5ec-30c9-4246-a450-d58e64703904 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.269854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daf80f-b18b-4aa5-9858-fe931258dceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.270485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daf80f-b18b-4aa5-9858-fe931258dceb None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.270761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daf80f-b18b-4aa5-9858-fe931258dceb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.271031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daf80f-b18b-4aa5-9858-fe931258dceb None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.271696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9daf80f-b18b-4aa5-9858-fe931258dceb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:37.272028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9daf80f-b18b-4aa5-9858-fe931258dceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.273404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9daf80f-b18b-4aa5-9858-fe931258dceb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.281604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 154/303] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:37.284062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6a6eb-5a67-4058-9150-815ebb3fe292 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.284408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6a6eb-5a67-4058-9150-815ebb3fe292 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.284768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6a6eb-5a67-4058-9150-815ebb3fe292 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.286348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 155/304] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:37.287036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6a6eb-5a67-4058-9150-815ebb3fe292 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.353642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019a5ec-30c9-4246-a450-d58e64703904 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.382255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:37.388428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65453865-8522-42b6-a9c2-6382d46a8540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.389055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65453865-8522-42b6-a9c2-6382d46a8540 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.389326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65453865-8522-42b6-a9c2-6382d46a8540 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.389589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65453865-8522-42b6-a9c2-6382d46a8540 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.390685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65453865-8522-42b6-a9c2-6382d46a8540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.392961 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65453865-8522-42b6-a9c2-6382d46a8540 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.394804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 150/305] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:37.395392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65453865-8522-42b6-a9c2-6382d46a8540 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.405162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6a6eb-5a67-4058-9150-815ebb3fe292 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.439866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16162ca-6498-41fc-bda8-d0b8b945b676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.440514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16162ca-6498-41fc-bda8-d0b8b945b676 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.440892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16162ca-6498-41fc-bda8-d0b8b945b676 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.441135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16162ca-6498-41fc-bda8-d0b8b945b676 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.441920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16162ca-6498-41fc-bda8-d0b8b945b676 None admin] RBAC: Adding query filter params (name=member) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:37.442219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16162ca-6498-41fc-bda8-d0b8b945b676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.443588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16162ca-6498-41fc-bda8-d0b8b945b676 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.445055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:37.453677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 156/306] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:37.455119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d505de-5aba-489b-9d70-55e7f23827b2 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.455392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d505de-5aba-489b-9d70-55e7f23827b2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.455697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d505de-5aba-489b-9d70-55e7f23827b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.473155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d0d505de-5aba-489b-9d70-55e7f23827b2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:37.474339 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d0d505de-5aba-489b-9d70-55e7f23827b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:37.502031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65453865-8522-42b6-a9c2-6382d46a8540 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.536470 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca713bf-4c14-435d-9c79-1b44ecc20767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.537107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca713bf-4c14-435d-9c79-1b44ecc20767 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.537383 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca713bf-4c14-435d-9c79-1b44ecc20767 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.537653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca713bf-4c14-435d-9c79-1b44ecc20767 None admin] PATH_INFO: `/v3/users/f09564ab4a76450db64287ee75e6d71b` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.542797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 157/307] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:37.544304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d505de-5aba-489b-9d70-55e7f23827b2 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.553845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca713bf-4c14-435d-9c79-1b44ecc20767 None admin] RBAC: Authorizing `identity:get_user(user_id=f09564ab4a76450db64287ee75e6d71b)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.555309 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca713bf-4c14-435d-9c79-1b44ecc20767 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.557157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 151/308] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/users/f09564ab4a76450db64287ee75e6d71b => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:37.560870 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca713bf-4c14-435d-9c79-1b44ecc20767 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.628795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:37.651892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d505de-5aba-489b-9d70-55e7f23827b2 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.683240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:37.683434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca713bf-4c14-435d-9c79-1b44ecc20767 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.699503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff549f3-0553-429a-bfb5-b752aeb9d666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.700299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff549f3-0553-429a-bfb5-b752aeb9d666 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.700456 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff549f3-0553-429a-bfb5-b752aeb9d666 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.700723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff549f3-0553-429a-bfb5-b752aeb9d666 None admin] PATH_INFO: `/v3/groups` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.701380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff549f3-0553-429a-bfb5-b752aeb9d666 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.702872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff549f3-0553-429a-bfb5-b752aeb9d666 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.722704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 158/309] 10.222.0.44 () {60 vars in 1134 bytes} [Tue Aug 30 01:02:37 2022] POST /identity/v3/groups => generated 229 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:02:37.723869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff549f3-0553-429a-bfb5-b752aeb9d666 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.735321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d110de4-2e35-4e9d-b549-4769871e3b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.737069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d110de4-2e35-4e9d-b549-4769871e3b45 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.737566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d110de4-2e35-4e9d-b549-4769871e3b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.738106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d110de4-2e35-4e9d-b549-4769871e3b45 None admin] PATH_INFO: `/v3/users/4c1073563c9f48eb9c8458e6f35aa456` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.739806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d110de4-2e35-4e9d-b549-4769871e3b45 None admin] RBAC: Authorizing `identity:get_user(user_id=4c1073563c9f48eb9c8458e6f35aa456)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.741881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d110de4-2e35-4e9d-b549-4769871e3b45 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.744701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 152/310] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/users/4c1073563c9f48eb9c8458e6f35aa456 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:37.750319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d110de4-2e35-4e9d-b549-4769871e3b45 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.874763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff549f3-0553-429a-bfb5-b752aeb9d666 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.890051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d110de4-2e35-4e9d-b549-4769871e3b45 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:37.925252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:37.926148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97273def-945a-478c-9ec4-dc55ecf3b5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.926816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97273def-945a-478c-9ec4-dc55ecf3b5be None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.927132 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97273def-945a-478c-9ec4-dc55ecf3b5be None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.927489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97273def-945a-478c-9ec4-dc55ecf3b5be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.928585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97273def-945a-478c-9ec4-dc55ecf3b5be None admin] RBAC: Adding query filter params (role.id=cd6b754c94e54de6b3bc5e17618f1e67, user.id=f09564ab4a76450db64287ee75e6d71b, scope.system=all) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:37.928916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97273def-945a-478c-9ec4-dc55ecf3b5be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.930673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97273def-945a-478c-9ec4-dc55ecf3b5be None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.937094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 159/311] 10.222.0.44 () {56 vars in 1308 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/role_assignments?user.id=f09564ab4a76450db64287ee75e6d71b&scope.system=all&role.id=cd6b754c94e54de6b3bc5e17618f1e67 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:37.937928 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97273def-945a-478c-9ec4-dc55ecf3b5be None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:37.940492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026142c9-ef33-424d-b9f0-1c6fe95fc6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:37.941578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026142c9-ef33-424d-b9f0-1c6fe95fc6cb None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:37.941856 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026142c9-ef33-424d-b9f0-1c6fe95fc6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:37.942240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026142c9-ef33-424d-b9f0-1c6fe95fc6cb None admin] PATH_INFO: `/v3/system/users/4c1073563c9f48eb9c8458e6f35aa456/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:37.943780 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026142c9-ef33-424d-b9f0-1c6fe95fc6cb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4c1073563c9f48eb9c8458e6f35aa456, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:37.945248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026142c9-ef33-424d-b9f0-1c6fe95fc6cb None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:37.963035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 153/312] 10.222.0.44 () {58 vars in 1334 bytes} [Tue Aug 30 01:02:37 2022] PUT /identity/v3/system/users/4c1073563c9f48eb9c8458e6f35aa456/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:38.009590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:38.082383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97273def-945a-478c-9ec4-dc55ecf3b5be None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:38.127315 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0e6be-b582-4746-a795-911495d5c424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:38.127850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0e6be-b582-4746-a795-911495d5c424 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.128159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0e6be-b582-4746-a795-911495d5c424 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.128425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0e6be-b582-4746-a795-911495d5c424 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.140284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0e6be-b582-4746-a795-911495d5c424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-b3c0e6be-b582-4746-a795-911495d5c424 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:38.145717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:38.146661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:38.147904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 160/313] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:38.153373 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026142c9-ef33-424d-b9f0-1c6fe95fc6cb None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:38.279596 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026142c9-ef33-424d-b9f0-1c6fe95fc6cb None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:38.325548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc00e64-7d8a-4614-b880-b881f381072e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:38.326160 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc00e64-7d8a-4614-b880-b881f381072e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.326478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc00e64-7d8a-4614-b880-b881f381072e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.326753 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc00e64-7d8a-4614-b880-b881f381072e None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.327419 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc00e64-7d8a-4614-b880-b881f381072e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:38.327721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc00e64-7d8a-4614-b880-b881f381072e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:38.329132 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc00e64-7d8a-4614-b880-b881f381072e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:38.342218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 154/314] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:38.348513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0e6be-b582-4746-a795-911495d5c424 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:38.506166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0e6be-b582-4746-a795-911495d5c424 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:38.525273 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:38.566959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb208b6-cbf5-4851-bd01-5e0eda7e7f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:38.568101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb208b6-cbf5-4851-bd01-5e0eda7e7f64 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.568629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb208b6-cbf5-4851-bd01-5e0eda7e7f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.569108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb208b6-cbf5-4851-bd01-5e0eda7e7f64 None admin] PATH_INFO: `/v3/users/32ff4ee17e2f45fc96ed4562ec48e88b` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.571053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb208b6-cbf5-4851-bd01-5e0eda7e7f64 None admin] RBAC: Authorizing `identity:get_user(user_id=32ff4ee17e2f45fc96ed4562ec48e88b)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:38.573690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb208b6-cbf5-4851-bd01-5e0eda7e7f64 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:38.577110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 161/315] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:38 2022] GET /identity/v3/users/32ff4ee17e2f45fc96ed4562ec48e88b => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:38.583241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc00e64-7d8a-4614-b880-b881f381072e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:38.639548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:38.693571 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc00e64-7d8a-4614-b880-b881f381072e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:38.728462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d981e-5cd2-4ed1-b03e-4285e1ef1441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:38.729064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d981e-5cd2-4ed1-b03e-4285e1ef1441 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.729338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d981e-5cd2-4ed1-b03e-4285e1ef1441 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.729604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d981e-5cd2-4ed1-b03e-4285e1ef1441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.797571 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d981e-5cd2-4ed1-b03e-4285e1ef1441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:38.799039 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d981e-5cd2-4ed1-b03e-4285e1ef1441 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:38.847904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:38.898735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 155/316] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:38.905674 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb208b6-cbf5-4851-bd01-5e0eda7e7f64 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:39.015284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb208b6-cbf5-4851-bd01-5e0eda7e7f64 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:39.053172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76756f66-7fc3-464d-adbd-de3c89fd56f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:39.053714 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76756f66-7fc3-464d-adbd-de3c89fd56f8 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:39.053987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76756f66-7fc3-464d-adbd-de3c89fd56f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:39.054255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76756f66-7fc3-464d-adbd-de3c89fd56f8 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:39.055375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76756f66-7fc3-464d-adbd-de3c89fd56f8 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:39.056857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76756f66-7fc3-464d-adbd-de3c89fd56f8 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:39.058785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 162/317] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:38 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:39.065221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d981e-5cd2-4ed1-b03e-4285e1ef1441 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:39.180996 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d981e-5cd2-4ed1-b03e-4285e1ef1441 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:39.217761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b28d70-8e90-42bf-956f-ae019daa440c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:39.218724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b28d70-8e90-42bf-956f-ae019daa440c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:39.219219 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b28d70-8e90-42bf-956f-ae019daa440c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:39.219678 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b28d70-8e90-42bf-956f-ae019daa440c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:39.220717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b28d70-8e90-42bf-956f-ae019daa440c None admin] RBAC: Adding query filter params (role.id=cd6b754c94e54de6b3bc5e17618f1e67, scope.project.id=dcbc59eca63d4c3ea02d6e944948c888, user.id=32ff4ee17e2f45fc96ed4562ec48e88b) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:39.221228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b28d70-8e90-42bf-956f-ae019daa440c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:39.222875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b28d70-8e90-42bf-956f-ae019daa440c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:39.230488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 156/318] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:02:39 2022] GET /identity/v3/role_assignments?user.id=32ff4ee17e2f45fc96ed4562ec48e88b&scope.project.id=dcbc59eca63d4c3ea02d6e944948c888&role.id=cd6b754c94e54de6b3bc5e17618f1e67 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:39.904110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3de2a4-6a21-48a0-82f4-294bfbabaa1c None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:39.904640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3de2a4-6a21-48a0-82f4-294bfbabaa1c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:39.904640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3de2a4-6a21-48a0-82f4-294bfbabaa1c None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:39.906528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 163/319] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:39.911716 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c83e0-f46e-4e1d-ad03-c2700f37251e None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:39.912209 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c83e0-f46e-4e1d-ad03-c2700f37251e None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:39.912280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c83e0-f46e-4e1d-ad03-c2700f37251e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:39.931773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-904c83e0-f46e-4e1d-ad03-c2700f37251e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:39.932974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-904c83e0-f46e-4e1d-ad03-c2700f37251e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:39.943471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:40.003357 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a0ed8-ccf0-436d-b4c7-0b87868aae62 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.003647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a0ed8-ccf0-436d-b4c7-0b87868aae62 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.003952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a0ed8-ccf0-436d-b4c7-0b87868aae62 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.005232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 157/320] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:40.005468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 164/321] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:02:40.010872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bdcbe-2f0b-4782-bf6c-c62a04c1ede5 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.011179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bdcbe-2f0b-4782-bf6c-c62a04c1ede5 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.011477 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bdcbe-2f0b-4782-bf6c-c62a04c1ede5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.031485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-de3bdcbe-2f0b-4782-bf6c-c62a04c1ede5 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:40.033510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-de3bdcbe-2f0b-4782-bf6c-c62a04c1ede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:40.084551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d0dc7-5381-4612-90f5-6c97be014273 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.085012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d0dc7-5381-4612-90f5-6c97be014273 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.085119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d0dc7-5381-4612-90f5-6c97be014273 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.086578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 165/322] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:40.091334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aed719-df87-4e54-9139-dc6237b6bfda None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.091608 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aed719-df87-4e54-9139-dc6237b6bfda None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.091883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aed719-df87-4e54-9139-dc6237b6bfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.108043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 158/323] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:40.114012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-21aed719-df87-4e54-9139-dc6237b6bfda None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:40.115232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-21aed719-df87-4e54-9139-dc6237b6bfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:40.161352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:40.186623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 166/324] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:02:40.273104 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bdcbe-2f0b-4782-bf6c-c62a04c1ede5 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:40.346462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:40.375182 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bdcbe-2f0b-4782-bf6c-c62a04c1ede5 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:40.395550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aed719-df87-4e54-9139-dc6237b6bfda None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:40.419733 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e4141-1700-4f7a-8037-d86b54631874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:40.421038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e4141-1700-4f7a-8037-d86b54631874 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.421675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e4141-1700-4f7a-8037-d86b54631874 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.421941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e4141-1700-4f7a-8037-d86b54631874 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.425018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e4141-1700-4f7a-8037-d86b54631874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:40.425948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e4141-1700-4f7a-8037-d86b54631874 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:40.427611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 159/325] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:02:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:40.433629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e4141-1700-4f7a-8037-d86b54631874 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:40.551571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aed719-df87-4e54-9139-dc6237b6bfda None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:40.595141 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e4f1e-9790-44e7-b6c1-5a16225a21e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:40.595786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e4f1e-9790-44e7-b6c1-5a16225a21e4 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.596090 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e4f1e-9790-44e7-b6c1-5a16225a21e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.596365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e4f1e-9790-44e7-b6c1-5a16225a21e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.607048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e4141-1700-4f7a-8037-d86b54631874 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:40.607767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e4f1e-9790-44e7-b6c1-5a16225a21e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-403e4f1e-9790-44e7-b6c1-5a16225a21e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:40.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:40.614643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:40.615625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 167/326] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:02:40.616360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e4f1e-9790-44e7-b6c1-5a16225a21e4 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:40.651104 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f38ab-f042-4764-8c26-c2aaa02f1564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:40.652354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f38ab-f042-4764-8c26-c2aaa02f1564 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.652354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f38ab-f042-4764-8c26-c2aaa02f1564 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.652354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f38ab-f042-4764-8c26-c2aaa02f1564 None admin] PATH_INFO: `/v3/groups` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.652922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f38ab-f042-4764-8c26-c2aaa02f1564 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:40.654354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f38ab-f042-4764-8c26-c2aaa02f1564 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:40.655852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:40.674155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 160/327] 10.222.0.44 () {60 vars in 1134 bytes} [Tue Aug 30 01:02:40 2022] POST /identity/v3/groups => generated 226 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:02:40.675099 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f38ab-f042-4764-8c26-c2aaa02f1564 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:40.678050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:40.765868 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e4f1e-9790-44e7-b6c1-5a16225a21e4 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:40.817779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f38ab-f042-4764-8c26-c2aaa02f1564 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:40.826033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082eb12d-7f9c-4024-a721-c7f68697cc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:40.826722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082eb12d-7f9c-4024-a721-c7f68697cc5e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.827011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082eb12d-7f9c-4024-a721-c7f68697cc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.827313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082eb12d-7f9c-4024-a721-c7f68697cc5e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.846296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082eb12d-7f9c-4024-a721-c7f68697cc5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-082eb12d-7f9c-4024-a721-c7f68697cc5e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:40.851287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:40.852481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:40.853343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:40.853343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:40.853343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:40.853343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:40.853343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:40.853343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:40.853343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:40.853343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:40.853343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:40.853343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:40.853343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:40.853343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:40.853979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 168/328] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:40.861038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082eb12d-7f9c-4024-a721-c7f68697cc5e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:40.866279 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e1365-2b3b-46a0-b814-4255b1f226e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:40.867145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e1365-2b3b-46a0-b814-4255b1f226e3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:40.867426 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e1365-2b3b-46a0-b814-4255b1f226e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:40.868671 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e1365-2b3b-46a0-b814-4255b1f226e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:40.869417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e1365-2b3b-46a0-b814-4255b1f226e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:40.869726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e1365-2b3b-46a0-b814-4255b1f226e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:40.871119 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e1365-2b3b-46a0-b814-4255b1f226e3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:40.883245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 161/329] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:40.887708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e1365-2b3b-46a0-b814-4255b1f226e3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:40.980068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:41.020787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:41.030895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082eb12d-7f9c-4024-a721-c7f68697cc5e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.056125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e1365-2b3b-46a0-b814-4255b1f226e3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.085728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4583f1-e0f2-44ec-83b5-456a6a392cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.086992 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4583f1-e0f2-44ec-83b5-456a6a392cba None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.087786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4583f1-e0f2-44ec-83b5-456a6a392cba None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.088633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4583f1-e0f2-44ec-83b5-456a6a392cba None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.090109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4583f1-e0f2-44ec-83b5-456a6a392cba None admin] RBAC: Adding query filter params (name=reader) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:41.091215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4583f1-e0f2-44ec-83b5-456a6a392cba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.094228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4583f1-e0f2-44ec-83b5-456a6a392cba None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.104059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684146a7-69d8-4846-85a1-e942da8ab6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.104713 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684146a7-69d8-4846-85a1-e942da8ab6ad None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.104992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684146a7-69d8-4846-85a1-e942da8ab6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.105264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684146a7-69d8-4846-85a1-e942da8ab6ad None admin] PATH_INFO: `/v3/users/4c1073563c9f48eb9c8458e6f35aa456` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.106517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684146a7-69d8-4846-85a1-e942da8ab6ad None admin] RBAC: Authorizing `identity:get_user(user_id=4c1073563c9f48eb9c8458e6f35aa456)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.107986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684146a7-69d8-4846-85a1-e942da8ab6ad None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.110365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 162/330] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:40 2022] GET /identity/v3/users/4c1073563c9f48eb9c8458e6f35aa456 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:41.116693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 169/331] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:41.117255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684146a7-69d8-4846-85a1-e942da8ab6ad None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.125024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4583f1-e0f2-44ec-83b5-456a6a392cba None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.287146 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684146a7-69d8-4846-85a1-e942da8ab6ad None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.312216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4583f1-e0f2-44ec-83b5-456a6a392cba None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.333959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:41.340790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f328e-5dd5-45eb-b6fb-81ccb928823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.341610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f328e-5dd5-45eb-b6fb-81ccb928823e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.341974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f328e-5dd5-45eb-b6fb-81ccb928823e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.342221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f328e-5dd5-45eb-b6fb-81ccb928823e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.342983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f328e-5dd5-45eb-b6fb-81ccb928823e None admin] RBAC: Adding query filter params (role.id=b07ea738cc204b2b8125bf4cd6d9c54a, user.id=4c1073563c9f48eb9c8458e6f35aa456, scope.system=all) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:41.343931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f328e-5dd5-45eb-b6fb-81ccb928823e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.346752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f328e-5dd5-45eb-b6fb-81ccb928823e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.364172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dea1c3-3412-4a33-bb28-b5c46f3ac80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.365550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dea1c3-3412-4a33-bb28-b5c46f3ac80f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.366253 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dea1c3-3412-4a33-bb28-b5c46f3ac80f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.366964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dea1c3-3412-4a33-bb28-b5c46f3ac80f None admin] PATH_INFO: `/v3/users/f09564ab4a76450db64287ee75e6d71b` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.368877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dea1c3-3412-4a33-bb28-b5c46f3ac80f None admin] RBAC: Authorizing `identity:get_user(user_id=f09564ab4a76450db64287ee75e6d71b)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.370238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 163/332] 10.222.0.44 () {56 vars in 1308 bytes} [Tue Aug 30 01:02:41 2022] GET /identity/v3/role_assignments?user.id=4c1073563c9f48eb9c8458e6f35aa456&scope.system=all&role.id=b07ea738cc204b2b8125bf4cd6d9c54a => generated 510 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:41.372090 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dea1c3-3412-4a33-bb28-b5c46f3ac80f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.376094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 170/333] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:41 2022] GET /identity/v3/users/f09564ab4a76450db64287ee75e6d71b => generated 293 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:41.385754 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f328e-5dd5-45eb-b6fb-81ccb928823e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.534118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f328e-5dd5-45eb-b6fb-81ccb928823e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:41.596398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b96a0-5b80-48c1-b6c6-d7027314ffdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:41.597350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b96a0-5b80-48c1-b6c6-d7027314ffdf None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.597797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b96a0-5b80-48c1-b6c6-d7027314ffdf None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.598264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b96a0-5b80-48c1-b6c6-d7027314ffdf None admin] PATH_INFO: `/v3/system/users/f09564ab4a76450db64287ee75e6d71b/roles/cd6b754c94e54de6b3bc5e17618f1e67` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.600557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b96a0-5b80-48c1-b6c6-d7027314ffdf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f09564ab4a76450db64287ee75e6d71b, role_id=cd6b754c94e54de6b3bc5e17618f1e67)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:41.602914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b96a0-5b80-48c1-b6c6-d7027314ffdf None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:41.615357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 164/334] 10.222.0.44 () {58 vars in 1334 bytes} [Tue Aug 30 01:02:41 2022] PUT /identity/v3/system/users/f09564ab4a76450db64287ee75e6d71b/roles/cd6b754c94e54de6b3bc5e17618f1e67 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:43.503481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fc3b2-f467-4b35-bf80-c4255e91fcf1 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:43.504332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fc3b2-f467-4b35-bf80-c4255e91fcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:43.504934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fc3b2-f467-4b35-bf80-c4255e91fcf1 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:43.507435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 171/335] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:43.518918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56482c21-555b-4df6-9e9d-71030752ef2c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:43.519700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56482c21-555b-4df6-9e9d-71030752ef2c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:43.519837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56482c21-555b-4df6-9e9d-71030752ef2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:43.559340 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-56482c21-555b-4df6-9e9d-71030752ef2c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:43.561460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-56482c21-555b-4df6-9e9d-71030752ef2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:43.668562 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:43.700679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 165/336] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:44.327359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e491b1-4141-465f-bb94-9cd7f7494d98 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.327851 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e491b1-4141-465f-bb94-9cd7f7494d98 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.328372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e491b1-4141-465f-bb94-9cd7f7494d98 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.330944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 172/337] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:44.337821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf94b6-d6ca-4cd7-9a79-84f6f01fb9c2 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.338270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf94b6-d6ca-4cd7-9a79-84f6f01fb9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.338695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf94b6-d6ca-4cd7-9a79-84f6f01fb9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.379100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-11cf94b6-d6ca-4cd7-9a79-84f6f01fb9c2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:44.381078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-11cf94b6-d6ca-4cd7-9a79-84f6f01fb9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:44.546355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 166/338] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:44.601446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e491b1-4141-465f-bb94-9cd7f7494d98 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.659745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:44.813948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e491b1-4141-465f-bb94-9cd7f7494d98 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.835956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7363678-3c24-4056-8a95-0d41c1803ee1 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.836300 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7363678-3c24-4056-8a95-0d41c1803ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.836810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7363678-3c24-4056-8a95-0d41c1803ee1 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.838346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 167/339] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:44.848394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b76526-452d-41e4-be38-97c4822f2256 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.848657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b76526-452d-41e4-be38-97c4822f2256 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.848965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b76526-452d-41e4-be38-97c4822f2256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.885611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2701e-79ae-4413-83aa-380a9fdb1bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:44.886776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2701e-79ae-4413-83aa-380a9fdb1bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.887358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2701e-79ae-4413-83aa-380a9fdb1bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.887969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2701e-79ae-4413-83aa-380a9fdb1bd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.903878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-24b76526-452d-41e4-be38-97c4822f2256 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:44.905829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-24b76526-452d-41e4-be38-97c4822f2256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:44.917833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2701e-79ae-4413-83aa-380a9fdb1bd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-65b2701e-79ae-4413-83aa-380a9fdb1bd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:44.935692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:44.936850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:44.938078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:44.938078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:44.938078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:44.938078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:44.938078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:44.938078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:44.938078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:44.938078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:44.938078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:44.938078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:44.938078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:44.938078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:44.939420 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 173/340] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:44.955209 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2701e-79ae-4413-83aa-380a9fdb1bd4 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:45.105460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 168/341] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:02:45.197225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:45.239173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2701e-79ae-4413-83aa-380a9fdb1bd4 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:45.291749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9103c3-4078-4e5b-87be-82ba0e56b7f0 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.292217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9103c3-4078-4e5b-87be-82ba0e56b7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.292638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9103c3-4078-4e5b-87be-82ba0e56b7f0 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.294555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 169/342] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:45.295739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9e19a-1857-47a2-858c-54d214d2bccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:45.296700 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9e19a-1857-47a2-858c-54d214d2bccb None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.297053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9e19a-1857-47a2-858c-54d214d2bccb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.297407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9e19a-1857-47a2-858c-54d214d2bccb None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.298856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9e19a-1857-47a2-858c-54d214d2bccb None admin] RBAC: Adding query filter params (name=member) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:45.299193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9e19a-1857-47a2-858c-54d214d2bccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:45.304729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69b985-a902-4c99-a182-0ca9967ece49 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.304729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69b985-a902-4c99-a182-0ca9967ece49 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.304729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69b985-a902-4c99-a182-0ca9967ece49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.308702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9e19a-1857-47a2-858c-54d214d2bccb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:45.332814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 174/343] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:45.337950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9e19a-1857-47a2-858c-54d214d2bccb None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:45.341899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-3d69b985-a902-4c99-a182-0ca9967ece49 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:45.346669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3d69b985-a902-4c99-a182-0ca9967ece49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:45.453989 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:45.517320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 170/344] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:45.518270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69b985-a902-4c99-a182-0ca9967ece49 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:45.587549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9e19a-1857-47a2-858c-54d214d2bccb None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:45.648987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452ccfd-3c55-48cf-aac1-724e86c2390c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:45.651053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452ccfd-3c55-48cf-aac1-724e86c2390c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.651799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452ccfd-3c55-48cf-aac1-724e86c2390c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.652363 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452ccfd-3c55-48cf-aac1-724e86c2390c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.678649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452ccfd-3c55-48cf-aac1-724e86c2390c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-f452ccfd-3c55-48cf-aac1-724e86c2390c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:45.700396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:45.702022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:45.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:45.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:45.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:45.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:45.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:45.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:45.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:45.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:45.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:45.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:45.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:45.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:45.705677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 175/345] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:45.717128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09aa04-de59-4d5a-9d78-00bbfb34f13d None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.717736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09aa04-de59-4d5a-9d78-00bbfb34f13d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.718295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09aa04-de59-4d5a-9d78-00bbfb34f13d None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.721177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 176/346] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:45 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:45.722666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09aa04-de59-4d5a-9d78-00bbfb34f13d None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:45.728652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69b985-a902-4c99-a182-0ca9967ece49 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:45.752380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:45.780594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57b7c-a0ad-4fd8-a6e8-5792f1430fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:45.781816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57b7c-a0ad-4fd8-a6e8-5792f1430fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.782340 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57b7c-a0ad-4fd8-a6e8-5792f1430fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.782944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57b7c-a0ad-4fd8-a6e8-5792f1430fe2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.841059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:45.874028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57b7c-a0ad-4fd8-a6e8-5792f1430fe2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:45.876771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57b7c-a0ad-4fd8-a6e8-5792f1430fe2 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:45.935903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09aa04-de59-4d5a-9d78-00bbfb34f13d None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:46.003831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98bf09-43c7-4382-a637-24d5dd38f58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:46.004774 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98bf09-43c7-4382-a637-24d5dd38f58c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:46.005664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98bf09-43c7-4382-a637-24d5dd38f58c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:46.005887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98bf09-43c7-4382-a637-24d5dd38f58c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:46.028811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 171/347] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:46.029777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98bf09-43c7-4382-a637-24d5dd38f58c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:46.030358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57b7c-a0ad-4fd8-a6e8-5792f1430fe2 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-1c98bf09-43c7-4382-a637-24d5dd38f58c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:46.042842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:46.044058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:46.044920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:46.044920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:46.044920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:46.044920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:46.044920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:46.044920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:46.044920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:46.044920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:46.044920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:46.044920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:46.044920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:46.044920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:46.046124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 177/348] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:02:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:46.050078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3ae6e-6e56-49fd-821d-49d27199147c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:46.050078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3ae6e-6e56-49fd-821d-49d27199147c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:46.050078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3ae6e-6e56-49fd-821d-49d27199147c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:46.084089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-28f3ae6e-6e56-49fd-821d-49d27199147c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:46.086416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-28f3ae6e-6e56-49fd-821d-49d27199147c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:46.165849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:46.200298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57b7c-a0ad-4fd8-a6e8-5792f1430fe2 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:46.200929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:46.208344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 178/349] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:46.209725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3ae6e-6e56-49fd-821d-49d27199147c None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:46.258785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29a0e1-7e5d-4ba4-848f-ad8128e9d146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:46.259320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29a0e1-7e5d-4ba4-848f-ad8128e9d146 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:46.259677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29a0e1-7e5d-4ba4-848f-ad8128e9d146 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:46.259944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29a0e1-7e5d-4ba4-848f-ad8128e9d146 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:46.260716 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29a0e1-7e5d-4ba4-848f-ad8128e9d146 None admin] RBAC: Adding query filter params (name=member) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:46.260959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29a0e1-7e5d-4ba4-848f-ad8128e9d146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:46.262338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29a0e1-7e5d-4ba4-848f-ad8128e9d146 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:46.278779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 172/350] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:46.281707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29a0e1-7e5d-4ba4-848f-ad8128e9d146 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:46.395736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3ae6e-6e56-49fd-821d-49d27199147c None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:46.456568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29a0e1-7e5d-4ba4-848f-ad8128e9d146 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:46.461279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4608ca-8960-4940-90cd-9a7a474c3531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:46.461653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4608ca-8960-4940-90cd-9a7a474c3531 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:46.462117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4608ca-8960-4940-90cd-9a7a474c3531 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:46.462557 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4608ca-8960-4940-90cd-9a7a474c3531 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:46.484033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4608ca-8960-4940-90cd-9a7a474c3531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-1a4608ca-8960-4940-90cd-9a7a474c3531 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:46.491433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:46.492458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:46.493787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 179/351] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:46.495641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4608ca-8960-4940-90cd-9a7a474c3531 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:46.515642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef8913-2ccb-47fa-a4f4-180d6f7078f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:46.518463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef8913-2ccb-47fa-a4f4-180d6f7078f4 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:46.519220 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef8913-2ccb-47fa-a4f4-180d6f7078f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:46.519848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef8913-2ccb-47fa-a4f4-180d6f7078f4 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:46.522340 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef8913-2ccb-47fa-a4f4-180d6f7078f4 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:46.524854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef8913-2ccb-47fa-a4f4-180d6f7078f4 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:46.532272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 173/352] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:46 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:46.534055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef8913-2ccb-47fa-a4f4-180d6f7078f4 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:46.540629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:46.613349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:46.700420 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4608ca-8960-4940-90cd-9a7a474c3531 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:46.739891 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef8913-2ccb-47fa-a4f4-180d6f7078f4 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:46.765054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a9653-56c9-4359-9ad6-3ef79248443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:46.765330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a9653-56c9-4359-9ad6-3ef79248443d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:46.765492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a9653-56c9-4359-9ad6-3ef79248443d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:46.765930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a9653-56c9-4359-9ad6-3ef79248443d None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:46.767079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a9653-56c9-4359-9ad6-3ef79248443d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:46.767079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a9653-56c9-4359-9ad6-3ef79248443d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:46.768742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a9653-56c9-4359-9ad6-3ef79248443d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:46.782963 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 180/353] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:02:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:46.784708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a9653-56c9-4359-9ad6-3ef79248443d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:46.809579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7d05e-46f5-4bff-a9aa-d79d2c2603a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:46.810164 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7d05e-46f5-4bff-a9aa-d79d2c2603a3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:46.810477 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7d05e-46f5-4bff-a9aa-d79d2c2603a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:46.810737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7d05e-46f5-4bff-a9aa-d79d2c2603a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:46.930073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7d05e-46f5-4bff-a9aa-d79d2c2603a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:46.931738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7d05e-46f5-4bff-a9aa-d79d2c2603a3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:46.935352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:46.964103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a9653-56c9-4359-9ad6-3ef79248443d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.003354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:47.037668 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33a509-fa11-4b66-ad1e-9a36bd18c6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.038241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33a509-fa11-4b66-ad1e-9a36bd18c6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.038621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33a509-fa11-4b66-ad1e-9a36bd18c6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.038780 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33a509-fa11-4b66-ad1e-9a36bd18c6f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.039467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33a509-fa11-4b66-ad1e-9a36bd18c6f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:47.039772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33a509-fa11-4b66-ad1e-9a36bd18c6f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.043406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33a509-fa11-4b66-ad1e-9a36bd18c6f0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.053739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 181/354] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:47.054609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33a509-fa11-4b66-ad1e-9a36bd18c6f0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.129059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 174/355] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:47.130089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7d05e-46f5-4bff-a9aa-d79d2c2603a3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.219527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33a509-fa11-4b66-ad1e-9a36bd18c6f0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.285991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d9ebf-6d22-4200-a84b-469691919cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.287212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d9ebf-6d22-4200-a84b-469691919cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.288741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d9ebf-6d22-4200-a84b-469691919cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.291115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d9ebf-6d22-4200-a84b-469691919cd2 None admin] PATH_INFO: `/v3/groups/d90461ea6cfc4d41837a8a536af051b4` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.303745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d9ebf-6d22-4200-a84b-469691919cd2 None admin] RBAC: Authorizing `identity:get_group(group_id=d90461ea6cfc4d41837a8a536af051b4)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.305863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d9ebf-6d22-4200-a84b-469691919cd2 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.308142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 182/356] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:02:47 2022] GET /identity/v3/groups/d90461ea6cfc4d41837a8a536af051b4 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:47.308697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d9ebf-6d22-4200-a84b-469691919cd2 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.309491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7d05e-46f5-4bff-a9aa-d79d2c2603a3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.322003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:47.366909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e741b-a881-440e-bac3-8d006d306def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.368147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e741b-a881-440e-bac3-8d006d306def None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.368584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e741b-a881-440e-bac3-8d006d306def None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.369035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e741b-a881-440e-bac3-8d006d306def None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.384907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e741b-a881-440e-bac3-8d006d306def None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-e28e741b-a881-440e-bac3-8d006d306def None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:47.395982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:47.397111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:47.397993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:47.397993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:47.397993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:47.397993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:47.397993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:47.397993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:47.397993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:47.397993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:47.397993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:47.397993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:47.397993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:47.397993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:47.398612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:47.399544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 175/357] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:02:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:47.401943 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c2005-c993-463c-8804-50e641c605e2 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.402410 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c2005-c993-463c-8804-50e641c605e2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.402823 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c2005-c993-463c-8804-50e641c605e2 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.405122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 176/358] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:47.406398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c2005-c993-463c-8804-50e641c605e2 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.464363 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d9ebf-6d22-4200-a84b-469691919cd2 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.512450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca239c-577d-412f-8da7-1d7d5ded313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.513784 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca239c-577d-412f-8da7-1d7d5ded313b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.514066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca239c-577d-412f-8da7-1d7d5ded313b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.514370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca239c-577d-412f-8da7-1d7d5ded313b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.606932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c2005-c993-463c-8804-50e641c605e2 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.642140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca239c-577d-412f-8da7-1d7d5ded313b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.643195 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca239c-577d-412f-8da7-1d7d5ded313b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.666828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007038be-02cc-4a9d-94f4-41a1587eacad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.667954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007038be-02cc-4a9d-94f4-41a1587eacad None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.668473 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007038be-02cc-4a9d-94f4-41a1587eacad None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.669157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007038be-02cc-4a9d-94f4-41a1587eacad None admin] PATH_INFO: `/v3/users/f09564ab4a76450db64287ee75e6d71b` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.673463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007038be-02cc-4a9d-94f4-41a1587eacad None admin] RBAC: Authorizing `identity:get_user(user_id=f09564ab4a76450db64287ee75e6d71b)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.675326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007038be-02cc-4a9d-94f4-41a1587eacad None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.677083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 177/359] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:02:47 2022] GET /identity/v3/users/f09564ab4a76450db64287ee75e6d71b => generated 293 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:47.677978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007038be-02cc-4a9d-94f4-41a1587eacad None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.721536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:47.729240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:47.823829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007038be-02cc-4a9d-94f4-41a1587eacad None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:47.830677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 183/360] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:47.830677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca239c-577d-412f-8da7-1d7d5ded313b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.884044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ae1ec-7ae1-47c0-a135-5b375542ba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:47.884630 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ae1ec-7ae1-47c0-a135-5b375542ba1a None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.884907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ae1ec-7ae1-47c0-a135-5b375542ba1a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.885177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ae1ec-7ae1-47c0-a135-5b375542ba1a None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.886408 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ae1ec-7ae1-47c0-a135-5b375542ba1a None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:47.887835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ae1ec-7ae1-47c0-a135-5b375542ba1a None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:47.889611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 178/361] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:47 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:47.890840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ae1ec-7ae1-47c0-a135-5b375542ba1a None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:47.994713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca239c-577d-412f-8da7-1d7d5ded313b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.041120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:48.047274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ae1ec-7ae1-47c0-a135-5b375542ba1a None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.052165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee53621-7031-4a53-970f-5795a62c73f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.052976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee53621-7031-4a53-970f-5795a62c73f5 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.053310 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee53621-7031-4a53-970f-5795a62c73f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.053635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee53621-7031-4a53-970f-5795a62c73f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.054466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee53621-7031-4a53-970f-5795a62c73f5 None admin] RBAC: Adding query filter params (group.id=d90461ea6cfc4d41837a8a536af051b4, role.id=b07ea738cc204b2b8125bf4cd6d9c54a, scope.project.id=8a6d5bbd6e874220b0d317fa4794df82) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:48.054836 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee53621-7031-4a53-970f-5795a62c73f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.056481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee53621-7031-4a53-970f-5795a62c73f5 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.064502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 184/362] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Aug 30 01:02:47 2022] GET /identity/v3/role_assignments?group.id=d90461ea6cfc4d41837a8a536af051b4&scope.project.id=8a6d5bbd6e874220b0d317fa4794df82&role.id=b07ea738cc204b2b8125bf4cd6d9c54a => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:48.066038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1f65e-65d5-4eb1-9cba-f4a3c8ca7eee None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.066512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1f65e-65d5-4eb1-9cba-f4a3c8ca7eee None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.066952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1f65e-65d5-4eb1-9cba-f4a3c8ca7eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.093260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-82d1f65e-65d5-4eb1-9cba-f4a3c8ca7eee None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:48.095312 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-82d1f65e-65d5-4eb1-9cba-f4a3c8ca7eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:48.099912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:48.112682 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fb7bf-99e7-4c5d-9075-212f4f135f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.114048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fb7bf-99e7-4c5d-9075-212f4f135f0f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.114505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fb7bf-99e7-4c5d-9075-212f4f135f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.114898 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fb7bf-99e7-4c5d-9075-212f4f135f0f None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.116434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fb7bf-99e7-4c5d-9075-212f4f135f0f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:48.117675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fb7bf-99e7-4c5d-9075-212f4f135f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.121201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fb7bf-99e7-4c5d-9075-212f4f135f0f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.136922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 179/363] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:02:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:48.138106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fb7bf-99e7-4c5d-9075-212f4f135f0f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.201573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 185/364] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:48.202729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1f65e-65d5-4eb1-9cba-f4a3c8ca7eee None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.340041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fb7bf-99e7-4c5d-9075-212f4f135f0f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.397428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1f65e-65d5-4eb1-9cba-f4a3c8ca7eee None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.402121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46baf4-9b73-4a04-8f91-b1b0a14ba39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.403074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46baf4-9b73-4a04-8f91-b1b0a14ba39b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.403537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46baf4-9b73-4a04-8f91-b1b0a14ba39b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.403984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46baf4-9b73-4a04-8f91-b1b0a14ba39b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.405334 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46baf4-9b73-4a04-8f91-b1b0a14ba39b None admin] RBAC: Adding query filter params (role.id=cd6b754c94e54de6b3bc5e17618f1e67, user.id=f09564ab4a76450db64287ee75e6d71b, scope.system=all) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:48.405864 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46baf4-9b73-4a04-8f91-b1b0a14ba39b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.408427 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46baf4-9b73-4a04-8f91-b1b0a14ba39b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.430640 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 180/365] 10.222.0.44 () {56 vars in 1308 bytes} [Tue Aug 30 01:02:48 2022] GET /identity/v3/role_assignments?user.id=f09564ab4a76450db64287ee75e6d71b&scope.system=all&role.id=cd6b754c94e54de6b3bc5e17618f1e67 => generated 510 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:48.432314 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46baf4-9b73-4a04-8f91-b1b0a14ba39b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.473158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:48.503675 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b61a52-05d6-4299-a230-82e904424ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.504582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b61a52-05d6-4299-a230-82e904424ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.504870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b61a52-05d6-4299-a230-82e904424ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.505287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b61a52-05d6-4299-a230-82e904424ce1 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.506669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b61a52-05d6-4299-a230-82e904424ce1 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.508347 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b61a52-05d6-4299-a230-82e904424ce1 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.510252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 186/366] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:48 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:48.511081 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b61a52-05d6-4299-a230-82e904424ce1 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.530954 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:48.614238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46baf4-9b73-4a04-8f91-b1b0a14ba39b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.685925 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61c78a-8c83-4ffd-9ee7-07ce056c97eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.687355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61c78a-8c83-4ffd-9ee7-07ce056c97eb None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.687890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61c78a-8c83-4ffd-9ee7-07ce056c97eb None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.689598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61c78a-8c83-4ffd-9ee7-07ce056c97eb None admin] PATH_INFO: `/v3/groups/d90461ea6cfc4d41837a8a536af051b4` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.692911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61c78a-8c83-4ffd-9ee7-07ce056c97eb None admin] RBAC: Authorizing `identity:get_group(group_id=d90461ea6cfc4d41837a8a536af051b4)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.694486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61c78a-8c83-4ffd-9ee7-07ce056c97eb None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.696590 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 181/367] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:02:48 2022] GET /identity/v3/groups/d90461ea6cfc4d41837a8a536af051b4 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:48.697345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b61a52-05d6-4299-a230-82e904424ce1 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.702271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61c78a-8c83-4ffd-9ee7-07ce056c97eb None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.763520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a989b19-1f6b-4376-96cd-30e37f5ec660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.764491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a989b19-1f6b-4376-96cd-30e37f5ec660 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.764845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a989b19-1f6b-4376-96cd-30e37f5ec660 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.765232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a989b19-1f6b-4376-96cd-30e37f5ec660 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.866031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:48.894858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61c78a-8c83-4ffd-9ee7-07ce056c97eb None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.909166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a989b19-1f6b-4376-96cd-30e37f5ec660 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.914011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a989b19-1f6b-4376-96cd-30e37f5ec660 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.944456 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:48.952235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8d195-6ad2-40ad-ae0b-40db4ccb0dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.953185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8d195-6ad2-40ad-ae0b-40db4ccb0dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.953469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8d195-6ad2-40ad-ae0b-40db4ccb0dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.955032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8d195-6ad2-40ad-ae0b-40db4ccb0dc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.974497 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8d195-6ad2-40ad-ae0b-40db4ccb0dc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-7ac8d195-6ad2-40ad-ae0b-40db4ccb0dc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:48.987284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:48.989239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:48.991023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:48.991023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:48.991023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:48.991023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:48.991023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:48.991023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:48.991023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:48.991023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:48.991023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:48.991023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:48.991023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:48.991023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:48.994802 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 182/368] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:02:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:48.996797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8d195-6ad2-40ad-ae0b-40db4ccb0dc3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:49.120951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 187/369] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:49.120951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a989b19-1f6b-4376-96cd-30e37f5ec660 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:49.197600 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8d195-6ad2-40ad-ae0b-40db4ccb0dc3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:49.245875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff11e2-068f-4f8c-b0ce-3d18c1634782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:49.246512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff11e2-068f-4f8c-b0ce-3d18c1634782 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:49.246791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff11e2-068f-4f8c-b0ce-3d18c1634782 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:49.247059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff11e2-068f-4f8c-b0ce-3d18c1634782 None admin] PATH_INFO: `/v3/groups/d90461ea6cfc4d41837a8a536af051b4` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:49.248367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff11e2-068f-4f8c-b0ce-3d18c1634782 None admin] RBAC: Authorizing `identity:get_group(group_id=d90461ea6cfc4d41837a8a536af051b4)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:49.249821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff11e2-068f-4f8c-b0ce-3d18c1634782 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:49.258886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 183/370] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:02:48 2022] GET /identity/v3/groups/d90461ea6cfc4d41837a8a536af051b4 => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:49.258886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff11e2-068f-4f8c-b0ce-3d18c1634782 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:49.286367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:49.286708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a989b19-1f6b-4376-96cd-30e37f5ec660 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:49.299204 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:49.331934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49b57-4545-47b5-a5cd-021c5ee8db53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:49.332748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49b57-4545-47b5-a5cd-021c5ee8db53 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:49.333224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49b57-4545-47b5-a5cd-021c5ee8db53 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:49.333520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49b57-4545-47b5-a5cd-021c5ee8db53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:49.334333 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a49b57-4545-47b5-a5cd-021c5ee8db53 None admin] RBAC: Adding query filter params (group.id=d90461ea6cfc4d41837a8a536af051b4, role.id=b07ea738cc204b2b8125bf4cd6d9c54a, scope.project.id=dcbc59eca63d4c3ea02d6e944948c888) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:49.334665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a49b57-4545-47b5-a5cd-021c5ee8db53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:49.336319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a49b57-4545-47b5-a5cd-021c5ee8db53 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:49.343394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 188/371] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Aug 30 01:02:49 2022] GET /identity/v3/role_assignments?group.id=d90461ea6cfc4d41837a8a536af051b4&scope.project.id=dcbc59eca63d4c3ea02d6e944948c888&role.id=b07ea738cc204b2b8125bf4cd6d9c54a => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:49.344496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49b57-4545-47b5-a5cd-021c5ee8db53 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:49.477412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff11e2-068f-4f8c-b0ce-3d18c1634782 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:49.518492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49b57-4545-47b5-a5cd-021c5ee8db53 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:49.553639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36e2ef-4e77-4802-9318-3f279411a52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:49.554412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36e2ef-4e77-4802-9318-3f279411a52e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:49.554719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36e2ef-4e77-4802-9318-3f279411a52e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:49.555064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36e2ef-4e77-4802-9318-3f279411a52e None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:49.555811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36e2ef-4e77-4802-9318-3f279411a52e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:49.556182 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36e2ef-4e77-4802-9318-3f279411a52e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:49.558039 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36e2ef-4e77-4802-9318-3f279411a52e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:49.570884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 184/372] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:49.571785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36e2ef-4e77-4802-9318-3f279411a52e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:49.597050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3fc7-adec-424e-a6ae-3cea58806d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:49.598879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3fc7-adec-424e-a6ae-3cea58806d3f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:49.599178 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3fc7-adec-424e-a6ae-3cea58806d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:49.599480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3fc7-adec-424e-a6ae-3cea58806d3f None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:49.601276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac3fc7-adec-424e-a6ae-3cea58806d3f None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:49.603013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac3fc7-adec-424e-a6ae-3cea58806d3f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:49.604775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 189/373] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:49 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:49.605561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3fc7-adec-424e-a6ae-3cea58806d3f None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:49.693165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:49.712454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:49.732228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36e2ef-4e77-4802-9318-3f279411a52e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:49.756984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3fc7-adec-424e-a6ae-3cea58806d3f None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:49.777627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517e4f0-8b0c-4ef9-b9c6-cc0a2ec185df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:49.778722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517e4f0-8b0c-4ef9-b9c6-cc0a2ec185df None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:49.779265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517e4f0-8b0c-4ef9-b9c6-cc0a2ec185df None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:49.779790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517e4f0-8b0c-4ef9-b9c6-cc0a2ec185df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:49.781004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517e4f0-8b0c-4ef9-b9c6-cc0a2ec185df None admin] RBAC: Adding query filter params (group.id=d90461ea6cfc4d41837a8a536af051b4, role.id=7562948bd63d483cad68427819d434a9, scope.project.id=8a6d5bbd6e874220b0d317fa4794df82) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:49.781594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517e4f0-8b0c-4ef9-b9c6-cc0a2ec185df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:49.783971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517e4f0-8b0c-4ef9-b9c6-cc0a2ec185df None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:49.800753 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 185/374] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Aug 30 01:02:49 2022] GET /identity/v3/role_assignments?group.id=d90461ea6cfc4d41837a8a536af051b4&scope.project.id=8a6d5bbd6e874220b0d317fa4794df82&role.id=7562948bd63d483cad68427819d434a9 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:49.802253 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517e4f0-8b0c-4ef9-b9c6-cc0a2ec185df None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:49.820522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d385813-d97f-49bb-82b8-d5bcda823945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:49.820522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d385813-d97f-49bb-82b8-d5bcda823945 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:49.820522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d385813-d97f-49bb-82b8-d5bcda823945 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:49.821263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d385813-d97f-49bb-82b8-d5bcda823945 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:49.972743 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517e4f0-8b0c-4ef9-b9c6-cc0a2ec185df None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.007487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d385813-d97f-49bb-82b8-d5bcda823945 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.011409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d385813-d97f-49bb-82b8-d5bcda823945 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.021707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a26ee9-b60a-4658-b471-fca884989d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.022861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a26ee9-b60a-4658-b471-fca884989d8e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.023417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a26ee9-b60a-4658-b471-fca884989d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.024109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a26ee9-b60a-4658-b471-fca884989d8e None admin] PATH_INFO: `/v3/groups/d90461ea6cfc4d41837a8a536af051b4` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.025764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a26ee9-b60a-4658-b471-fca884989d8e None admin] RBAC: Authorizing `identity:get_group(group_id=d90461ea6cfc4d41837a8a536af051b4)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.028098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a26ee9-b60a-4658-b471-fca884989d8e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.030829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 186/375] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:02:49 2022] GET /identity/v3/groups/d90461ea6cfc4d41837a8a536af051b4 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:50.040831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a26ee9-b60a-4658-b471-fca884989d8e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.083744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:50.105517 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:50.258160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 190/376] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:50.264725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d385813-d97f-49bb-82b8-d5bcda823945 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.304219 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a26ee9-b60a-4658-b471-fca884989d8e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.369751 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6f19a-41a3-4cd7-9d39-e7714fd8319d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.370767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6f19a-41a3-4cd7-9d39-e7714fd8319d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.371243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6f19a-41a3-4cd7-9d39-e7714fd8319d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.371558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6f19a-41a3-4cd7-9d39-e7714fd8319d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.372901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a6f19a-41a3-4cd7-9d39-e7714fd8319d None admin] RBAC: Adding query filter params (group.id=d90461ea6cfc4d41837a8a536af051b4, role.id=7562948bd63d483cad68427819d434a9, scope.project.id=dcbc59eca63d4c3ea02d6e944948c888) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:50.373336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a6f19a-41a3-4cd7-9d39-e7714fd8319d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.375795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a6f19a-41a3-4cd7-9d39-e7714fd8319d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.397769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 187/377] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Aug 30 01:02:50 2022] GET /identity/v3/role_assignments?group.id=d90461ea6cfc4d41837a8a536af051b4&scope.project.id=dcbc59eca63d4c3ea02d6e944948c888&role.id=7562948bd63d483cad68427819d434a9 => generated 260 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:50.486388 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d385813-d97f-49bb-82b8-d5bcda823945 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.547189 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428497d5-534f-4441-a9e2-345fba49346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.548358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428497d5-534f-4441-a9e2-345fba49346b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.548707 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428497d5-534f-4441-a9e2-345fba49346b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.548980 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428497d5-534f-4441-a9e2-345fba49346b None admin] PATH_INFO: `/v3/projects/236d122dbb6b4283a260bcccbdcd4a4a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.550456 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428497d5-534f-4441-a9e2-345fba49346b None admin] RBAC: Authorizing `identity:get_project(project_id=236d122dbb6b4283a260bcccbdcd4a4a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.552029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428497d5-534f-4441-a9e2-345fba49346b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.554176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 191/378] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:50 2022] GET /identity/v3/projects/236d122dbb6b4283a260bcccbdcd4a4a => generated 344 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:50.560479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6f19a-41a3-4cd7-9d39-e7714fd8319d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.576047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6df3d1-11f5-4727-b771-b5599bfafc06 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.576395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6df3d1-11f5-4727-b771-b5599bfafc06 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.576605 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6df3d1-11f5-4727-b771-b5599bfafc06 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.578912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 192/379] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:50.585047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95fb36-87cf-442d-bf2e-bbe31a71f1e3 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.585418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95fb36-87cf-442d-bf2e-bbe31a71f1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.585605 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95fb36-87cf-442d-bf2e-bbe31a71f1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.619003 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-cf95fb36-87cf-442d-bf2e-bbe31a71f1e3 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:50.620350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-cf95fb36-87cf-442d-bf2e-bbe31a71f1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:50.629459 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:50.691889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:50.740443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 193/380] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:50.742524 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6f19a-41a3-4cd7-9d39-e7714fd8319d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.812025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60844545-7ff5-490a-ae36-f6022f11e4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:50.812612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60844545-7ff5-490a-ae36-f6022f11e4ed None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.812892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60844545-7ff5-490a-ae36-f6022f11e4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.813163 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60844545-7ff5-490a-ae36-f6022f11e4ed None admin] PATH_INFO: `/v3/groups/2079eaf7ffbc43eeb787b5f65841a2e6` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.824913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60844545-7ff5-490a-ae36-f6022f11e4ed None admin] RBAC: Authorizing `identity:get_group(group_id=2079eaf7ffbc43eeb787b5f65841a2e6)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:50.826808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60844545-7ff5-490a-ae36-f6022f11e4ed None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:50.828860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 188/381] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:02:50 2022] GET /identity/v3/groups/2079eaf7ffbc43eeb787b5f65841a2e6 => generated 229 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:50.836560 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95fb36-87cf-442d-bf2e-bbe31a71f1e3 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.988987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95fb36-87cf-442d-bf2e-bbe31a71f1e3 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.035536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d24f0-e44a-4598-88dd-be12f2b758ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.036708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d24f0-e44a-4598-88dd-be12f2b758ef None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.037349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d24f0-e44a-4598-88dd-be12f2b758ef None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.037937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d24f0-e44a-4598-88dd-be12f2b758ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.039091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d24f0-e44a-4598-88dd-be12f2b758ef None admin] RBAC: Adding query filter params (group.id=2079eaf7ffbc43eeb787b5f65841a2e6, role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=236d122dbb6b4283a260bcccbdcd4a4a) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:51.039635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d24f0-e44a-4598-88dd-be12f2b758ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.041391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d24f0-e44a-4598-88dd-be12f2b758ef None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.050464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 194/382] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Aug 30 01:02:50 2022] GET /identity/v3/role_assignments?group.id=2079eaf7ffbc43eeb787b5f65841a2e6&scope.project.id=236d122dbb6b4283a260bcccbdcd4a4a&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:51.169180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60844545-7ff5-490a-ae36-f6022f11e4ed None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.303076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60844545-7ff5-490a-ae36-f6022f11e4ed None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.342676 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c1a68-eec7-4cbd-95a8-9aa0f6866d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.343363 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c1a68-eec7-4cbd-95a8-9aa0f6866d69 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.343688 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c1a68-eec7-4cbd-95a8-9aa0f6866d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.343965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c1a68-eec7-4cbd-95a8-9aa0f6866d69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.352136 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:51.356706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c1a68-eec7-4cbd-95a8-9aa0f6866d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-c64c1a68-eec7-4cbd-95a8-9aa0f6866d69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:51.363402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:51.364739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:51.365910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 189/383] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:51.374462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d24f0-e44a-4598-88dd-be12f2b758ef None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.383525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:51.653430 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d24f0-e44a-4598-88dd-be12f2b758ef None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.730674 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71ae86-e052-4b5e-b0d4-009affa344ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.731424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71ae86-e052-4b5e-b0d4-009affa344ac None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.731762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71ae86-e052-4b5e-b0d4-009affa344ac None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.732061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71ae86-e052-4b5e-b0d4-009affa344ac None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.732812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71ae86-e052-4b5e-b0d4-009affa344ac None admin] RBAC: Adding query filter params (name=member) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:51.733122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71ae86-e052-4b5e-b0d4-009affa344ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.735112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71ae86-e052-4b5e-b0d4-009affa344ac None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.752573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 195/384] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:51.759108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c1a68-eec7-4cbd-95a8-9aa0f6866d69 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:51.938415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c1a68-eec7-4cbd-95a8-9aa0f6866d69 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:51.995721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1994e-d5c1-4bf5-8f61-ec3e6c61d362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.997010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1994e-d5c1-4bf5-8f61-ec3e6c61d362 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.997610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1994e-d5c1-4bf5-8f61-ec3e6c61d362 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.998231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1994e-d5c1-4bf5-8f61-ec3e6c61d362 None admin] PATH_INFO: `/v3/groups/d90461ea6cfc4d41837a8a536af051b4` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:52.005149 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1994e-d5c1-4bf5-8f61-ec3e6c61d362 None admin] RBAC: Authorizing `identity:get_group(group_id=d90461ea6cfc4d41837a8a536af051b4)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:52.007038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1994e-d5c1-4bf5-8f61-ec3e6c61d362 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:52.009137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 190/385] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:02:51 2022] GET /identity/v3/groups/d90461ea6cfc4d41837a8a536af051b4 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:52.015199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71ae86-e052-4b5e-b0d4-009affa344ac None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:52.142847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:52.205796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71ae86-e052-4b5e-b0d4-009affa344ac None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:52.220837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49101e85-0407-466c-9b9a-0e80c9c4578a None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:52.221501 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49101e85-0407-466c-9b9a-0e80c9c4578a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:52.221756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49101e85-0407-466c-9b9a-0e80c9c4578a None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:52.224095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 191/386] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:52.230769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02e131-4649-441d-8340-6d67f1d2b378 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:52.231061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02e131-4649-441d-8340-6d67f1d2b378 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:52.231361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02e131-4649-441d-8340-6d67f1d2b378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:52.265175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-3d02e131-4649-441d-8340-6d67f1d2b378 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:52.266485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3d02e131-4649-441d-8340-6d67f1d2b378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:52.289256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03207a-8128-4f65-b999-1e56d54ed3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:52.290317 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03207a-8128-4f65-b999-1e56d54ed3ec None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:52.290789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03207a-8128-4f65-b999-1e56d54ed3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:52.291262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03207a-8128-4f65-b999-1e56d54ed3ec None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:52.294639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03207a-8128-4f65-b999-1e56d54ed3ec None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:52.296493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03207a-8128-4f65-b999-1e56d54ed3ec None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:52.299475 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 196/387] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:52 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:52.307088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03207a-8128-4f65-b999-1e56d54ed3ec None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:52.444395 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:52.458700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 192/388] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:52.512436 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03207a-8128-4f65-b999-1e56d54ed3ec None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:52.553126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:52.563394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a2c8c-7fd3-4201-9b10-f0f604676947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:52.564117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a2c8c-7fd3-4201-9b10-f0f604676947 None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:52.564404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a2c8c-7fd3-4201-9b10-f0f604676947 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:52.564673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a2c8c-7fd3-4201-9b10-f0f604676947 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82/groups/d90461ea6cfc4d41837a8a536af051b4/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:52.569358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a2c8c-7fd3-4201-9b10-f0f604676947 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a6d5bbd6e874220b0d317fa4794df82, group_id=d90461ea6cfc4d41837a8a536af051b4, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:52.570885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a2c8c-7fd3-4201-9b10-f0f604676947 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:52.589096 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 197/389] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Aug 30 01:02:52 2022] PUT /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82/groups/d90461ea6cfc4d41837a8a536af051b4/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:52.811669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02e131-4649-441d-8340-6d67f1d2b378 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:52.930696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da767eb8-ec91-4567-b159-c0f3cd2801f6 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:52.930982 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da767eb8-ec91-4567-b159-c0f3cd2801f6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:52.931258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da767eb8-ec91-4567-b159-c0f3cd2801f6 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:52.932811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 198/390] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:52.939229 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283cb47-a1be-41ed-8d7e-c4ec6517459b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:52.939586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283cb47-a1be-41ed-8d7e-c4ec6517459b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:52.939826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283cb47-a1be-41ed-8d7e-c4ec6517459b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:52.973055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-5283cb47-a1be-41ed-8d7e-c4ec6517459b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:52.974185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5283cb47-a1be-41ed-8d7e-c4ec6517459b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:52.979894 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02e131-4649-441d-8340-6d67f1d2b378 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:53.064263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9acc8e-bd8b-4a5c-b335-6d449465568d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:53.064935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9acc8e-bd8b-4a5c-b335-6d449465568d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:53.065213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9acc8e-bd8b-4a5c-b335-6d449465568d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:53.065482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9acc8e-bd8b-4a5c-b335-6d449465568d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:53.078038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9acc8e-bd8b-4a5c-b335-6d449465568d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-ec9acc8e-bd8b-4a5c-b335-6d449465568d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:53.083577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:53.085295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:53.086928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 193/391] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:53.093714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9acc8e-bd8b-4a5c-b335-6d449465568d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:53.139930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 199/392] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:02:53.142215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4131582-f7e0-49ee-aaf2-cb419845e38a None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:53.142589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4131582-f7e0-49ee-aaf2-cb419845e38a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:53.142902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4131582-f7e0-49ee-aaf2-cb419845e38a None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:53.148927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 200/393] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:53.151060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e5867-16c5-4023-9b6d-c474092461d9 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:53.151343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e5867-16c5-4023-9b6d-c474092461d9 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:53.151611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e5867-16c5-4023-9b6d-c474092461d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:53.202026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-cf8e5867-16c5-4023-9b6d-c474092461d9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:53.203660 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-cf8e5867-16c5-4023-9b6d-c474092461d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:53.232642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:53.314143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9acc8e-bd8b-4a5c-b335-6d449465568d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:53.361606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 201/394] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:02:53.385564 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34459c2-5849-4d84-a95f-c299f7f015b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:53.386329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34459c2-5849-4d84-a95f-c299f7f015b1 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:53.386709 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34459c2-5849-4d84-a95f-c299f7f015b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:53.387010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34459c2-5849-4d84-a95f-c299f7f015b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:53.387804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34459c2-5849-4d84-a95f-c299f7f015b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:53.388239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34459c2-5849-4d84-a95f-c299f7f015b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:53.390223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34459c2-5849-4d84-a95f-c299f7f015b1 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:53.407890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 194/395] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:53.413775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e5867-16c5-4023-9b6d-c474092461d9 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:53.480151 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:53.499941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34459c2-5849-4d84-a95f-c299f7f015b1 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:53.540865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e5867-16c5-4023-9b6d-c474092461d9 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:53.582112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c841cb-450e-4302-968c-7949c876b2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:53.586180 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c841cb-450e-4302-968c-7949c876b2ae None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:53.586519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c841cb-450e-4302-968c-7949c876b2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:53.586795 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c841cb-450e-4302-968c-7949c876b2ae None admin] PATH_INFO: `/v3/groups/d90461ea6cfc4d41837a8a536af051b4` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:53.588089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c841cb-450e-4302-968c-7949c876b2ae None admin] RBAC: Authorizing `identity:get_group(group_id=d90461ea6cfc4d41837a8a536af051b4)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:53.589677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c841cb-450e-4302-968c-7949c876b2ae None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:53.591304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 202/396] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:02:53 2022] GET /identity/v3/groups/d90461ea6cfc4d41837a8a536af051b4 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:53.602146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c841cb-450e-4302-968c-7949c876b2ae None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:53.727760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34459c2-5849-4d84-a95f-c299f7f015b1 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:53.753892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:53.776621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71726f11-c188-466d-b0e1-af481c3a0fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:53.778010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71726f11-c188-466d-b0e1-af481c3a0fac None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:53.778558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71726f11-c188-466d-b0e1-af481c3a0fac None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:53.779056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71726f11-c188-466d-b0e1-af481c3a0fac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:53.801225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71726f11-c188-466d-b0e1-af481c3a0fac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:53.807504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c841cb-450e-4302-968c-7949c876b2ae None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-71726f11-c188-466d-b0e1-af481c3a0fac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:53.808391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:53.809354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:53.810295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:53.810295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:53.810295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:53.810295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:53.810295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:53.810295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:53.810295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:53.810295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:53.810295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:53.810295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:53.810295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:53.810295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:53.814932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 195/397] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:02:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:02:53.820035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71726f11-c188-466d-b0e1-af481c3a0fac None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:53.840564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:53.864369 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c93863-176c-430b-a0c9-53cfe46a16b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:53.865323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c93863-176c-430b-a0c9-53cfe46a16b4 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:53.865814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c93863-176c-430b-a0c9-53cfe46a16b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:53.866233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c93863-176c-430b-a0c9-53cfe46a16b4 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:53.868091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c93863-176c-430b-a0c9-53cfe46a16b4 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:53.869875 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c93863-176c-430b-a0c9-53cfe46a16b4 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:53.872617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 203/398] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:53 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:53.873711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c93863-176c-430b-a0c9-53cfe46a16b4 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:53.993175 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c93863-176c-430b-a0c9-53cfe46a16b4 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:54.016116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71726f11-c188-466d-b0e1-af481c3a0fac None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:54.038379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154b3c8-2306-4a10-bc1f-0fbd58491792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:54.039444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154b3c8-2306-4a10-bc1f-0fbd58491792 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:54.039850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154b3c8-2306-4a10-bc1f-0fbd58491792 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:54.040543 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154b3c8-2306-4a10-bc1f-0fbd58491792 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:54.041467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154b3c8-2306-4a10-bc1f-0fbd58491792 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:54.041896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154b3c8-2306-4a10-bc1f-0fbd58491792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:54.046393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154b3c8-2306-4a10-bc1f-0fbd58491792 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:54.064106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 204/399] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:02:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:54.064106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8e58a-3236-49d1-9e07-54ffbd5c0f5f None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:54.064106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8e58a-3236-49d1-9e07-54ffbd5c0f5f None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:54.064106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8e58a-3236-49d1-9e07-54ffbd5c0f5f None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:54.064624 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 205/400] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:54.066912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8e58a-3236-49d1-9e07-54ffbd5c0f5f None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:54.076558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cc88e-cb24-4ee2-8f68-d296eb1ea25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:54.078703 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cc88e-cb24-4ee2-8f68-d296eb1ea25c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:54.079472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cc88e-cb24-4ee2-8f68-d296eb1ea25c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:54.079944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cc88e-cb24-4ee2-8f68-d296eb1ea25c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:54.096885 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60cc88e-cb24-4ee2-8f68-d296eb1ea25c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-d60cc88e-cb24-4ee2-8f68-d296eb1ea25c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:54.102977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:54.103956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:54.104845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:54.104845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:54.104845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:54.104845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:54.104845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:54.104845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:54.104845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:54.104845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:54.104845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:54.104845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:54.104845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:54.104845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:54.109911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 196/401] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:02:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:54.111667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cc88e-cb24-4ee2-8f68-d296eb1ea25c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:54.136369 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:54.161822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:54.222437 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8e58a-3236-49d1-9e07-54ffbd5c0f5f None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:54.246842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cc88e-cb24-4ee2-8f68-d296eb1ea25c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:54.280293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bac8d-dab4-42dc-8de0-b3afcfdfba45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:54.281729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bac8d-dab4-42dc-8de0-b3afcfdfba45 None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:54.283655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bac8d-dab4-42dc-8de0-b3afcfdfba45 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:54.284565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bac8d-dab4-42dc-8de0-b3afcfdfba45 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888/groups/d90461ea6cfc4d41837a8a536af051b4/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:54.287942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bac8d-dab4-42dc-8de0-b3afcfdfba45 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcbc59eca63d4c3ea02d6e944948c888, group_id=d90461ea6cfc4d41837a8a536af051b4, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:54.290697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bac8d-dab4-42dc-8de0-b3afcfdfba45 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:54.290995 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1a149-5498-46cb-afa7-bbfcc475cda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:54.291614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1a149-5498-46cb-afa7-bbfcc475cda5 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:54.292083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1a149-5498-46cb-afa7-bbfcc475cda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:54.292569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1a149-5498-46cb-afa7-bbfcc475cda5 None admin] PATH_INFO: `/v3/groups/d90461ea6cfc4d41837a8a536af051b4` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:54.293942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1a149-5498-46cb-afa7-bbfcc475cda5 None admin] RBAC: Authorizing `identity:get_group(group_id=d90461ea6cfc4d41837a8a536af051b4)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:54.296539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1a149-5498-46cb-afa7-bbfcc475cda5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:54.299702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 197/402] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:02:54 2022] GET /identity/v3/groups/d90461ea6cfc4d41837a8a536af051b4 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:54.301993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf82e6f-3f4b-48dc-8c26-847083e6300f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:54.302441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf82e6f-3f4b-48dc-8c26-847083e6300f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:54.302852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf82e6f-3f4b-48dc-8c26-847083e6300f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:54.332116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 206/403] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Aug 30 01:02:54 2022] PUT /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888/groups/d90461ea6cfc4d41837a8a536af051b4/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:54.333667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bac8d-dab4-42dc-8de0-b3afcfdfba45 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:54.335315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-2bf82e6f-3f4b-48dc-8c26-847083e6300f None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:54.337526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2bf82e6f-3f4b-48dc-8c26-847083e6300f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:54.472013 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 198/404] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:54.473178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf82e6f-3f4b-48dc-8c26-847083e6300f None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:54.565903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bac8d-dab4-42dc-8de0-b3afcfdfba45 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:54.566532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:54.580232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:54.614640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d88b6-2813-4ee0-b5c8-b948561e7b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:54.615291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d88b6-2813-4ee0-b5c8-b948561e7b04 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:54.615568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d88b6-2813-4ee0-b5c8-b948561e7b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:54.615847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d88b6-2813-4ee0-b5c8-b948561e7b04 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:54.616597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d88b6-2813-4ee0-b5c8-b948561e7b04 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:54.616920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d88b6-2813-4ee0-b5c8-b948561e7b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:54.618483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d88b6-2813-4ee0-b5c8-b948561e7b04 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:54.631024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 207/405] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:02:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:54.638317 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d88b6-2813-4ee0-b5c8-b948561e7b04 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:54.678379 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf82e6f-3f4b-48dc-8c26-847083e6300f None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:54.780768 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b0dd6-15ea-495b-8b14-826e2bde42a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:54.781432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b0dd6-15ea-495b-8b14-826e2bde42a3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:54.781713 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b0dd6-15ea-495b-8b14-826e2bde42a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:54.782127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b0dd6-15ea-495b-8b14-826e2bde42a3 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:54.783506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b0dd6-15ea-495b-8b14-826e2bde42a3 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:54.785019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b0dd6-15ea-495b-8b14-826e2bde42a3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:54.788220 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 199/406] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:54 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:54.795988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b0dd6-15ea-495b-8b14-826e2bde42a3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:54.808768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d88b6-2813-4ee0-b5c8-b948561e7b04 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:54.855145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850793db-4f81-41cc-9c5c-6a8d6998b39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:54.856108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850793db-4f81-41cc-9c5c-6a8d6998b39e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:54.856590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850793db-4f81-41cc-9c5c-6a8d6998b39e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:54.857037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850793db-4f81-41cc-9c5c-6a8d6998b39e None admin] PATH_INFO: `/v3/groups/d90461ea6cfc4d41837a8a536af051b4` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:54.858428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850793db-4f81-41cc-9c5c-6a8d6998b39e None admin] RBAC: Authorizing `identity:get_group(group_id=d90461ea6cfc4d41837a8a536af051b4)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:54.860187 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850793db-4f81-41cc-9c5c-6a8d6998b39e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:54.863315 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 208/407] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:02:54 2022] GET /identity/v3/groups/d90461ea6cfc4d41837a8a536af051b4 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:54.874964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850793db-4f81-41cc-9c5c-6a8d6998b39e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:54.972742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:54.975643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b0dd6-15ea-495b-8b14-826e2bde42a3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:54.978358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:55.015308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7e156-a011-4760-86fc-37b524308dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:55.015981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7e156-a011-4760-86fc-37b524308dd7 None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:55.016798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7e156-a011-4760-86fc-37b524308dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:55.017077 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7e156-a011-4760-86fc-37b524308dd7 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82/groups/d90461ea6cfc4d41837a8a536af051b4/roles/7562948bd63d483cad68427819d434a9` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:55.019315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7e156-a011-4760-86fc-37b524308dd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a6d5bbd6e874220b0d317fa4794df82, group_id=d90461ea6cfc4d41837a8a536af051b4, role_id=7562948bd63d483cad68427819d434a9)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:55.020814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7e156-a011-4760-86fc-37b524308dd7 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:55.044033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 200/408] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Aug 30 01:02:54 2022] PUT /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82/groups/d90461ea6cfc4d41837a8a536af051b4/roles/7562948bd63d483cad68427819d434a9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:02:55.044033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7e156-a011-4760-86fc-37b524308dd7 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:55.072809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850793db-4f81-41cc-9c5c-6a8d6998b39e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:55.142287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df894c62-94e9-4f4a-af45-3f531f7aab26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:55.143008 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df894c62-94e9-4f4a-af45-3f531f7aab26 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:55.143293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df894c62-94e9-4f4a-af45-3f531f7aab26 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:55.143566 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df894c62-94e9-4f4a-af45-3f531f7aab26 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:55.145100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df894c62-94e9-4f4a-af45-3f531f7aab26 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:55.146820 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df894c62-94e9-4f4a-af45-3f531f7aab26 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:55.149289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 209/409] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:54 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:55.150735 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c564ae0-5314-4779-9329-3ff5a48e47ef None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:55.151016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c564ae0-5314-4779-9329-3ff5a48e47ef None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:55.151291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c564ae0-5314-4779-9329-3ff5a48e47ef None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:55.153368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 210/410] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:55.159384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c564ae0-5314-4779-9329-3ff5a48e47ef None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:55.180581 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7e156-a011-4760-86fc-37b524308dd7 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:55.227381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b02784-abd9-4785-ba17-486ec241cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:55.228521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b02784-abd9-4785-ba17-486ec241cb8e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:55.229118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b02784-abd9-4785-ba17-486ec241cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:55.229677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b02784-abd9-4785-ba17-486ec241cb8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:55.248243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b02784-abd9-4785-ba17-486ec241cb8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-91b02784-abd9-4785-ba17-486ec241cb8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:55.255495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:55.256693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:55.257565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 201/411] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:02:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:55.258422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43985c4b-0f65-4911-8e1b-72aa0c056d78 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:55.258771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43985c4b-0f65-4911-8e1b-72aa0c056d78 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:55.259048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43985c4b-0f65-4911-8e1b-72aa0c056d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:55.285629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-43985c4b-0f65-4911-8e1b-72aa0c056d78 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:55.287209 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-43985c4b-0f65-4911-8e1b-72aa0c056d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:55.313943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c564ae0-5314-4779-9329-3ff5a48e47ef None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:55.315699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:55.331531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:55.369330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9bcddc-a7ca-4541-9cc8-3cfc2c49430d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:55.371128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bcddc-a7ca-4541-9cc8-3cfc2c49430d None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:55.371617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bcddc-a7ca-4541-9cc8-3cfc2c49430d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:55.372379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bcddc-a7ca-4541-9cc8-3cfc2c49430d None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888/groups/d90461ea6cfc4d41837a8a536af051b4/roles/7562948bd63d483cad68427819d434a9` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:55.376266 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bcddc-a7ca-4541-9cc8-3cfc2c49430d None admin] RBAC: Authorizing `identity:create_grant(project_id=dcbc59eca63d4c3ea02d6e944948c888, group_id=d90461ea6cfc4d41837a8a536af051b4, role_id=7562948bd63d483cad68427819d434a9)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:55.378863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bcddc-a7ca-4541-9cc8-3cfc2c49430d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:55.383336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 202/412] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:55.384404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43985c4b-0f65-4911-8e1b-72aa0c056d78 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:55.397159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 211/413] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Aug 30 01:02:55 2022] PUT /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888/groups/d90461ea6cfc4d41837a8a536af051b4/roles/7562948bd63d483cad68427819d434a9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:55.528866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43985c4b-0f65-4911-8e1b-72aa0c056d78 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:55.578854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612e14b-b1b5-44d1-afbe-474bc91a0ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:55.579541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612e14b-b1b5-44d1-afbe-474bc91a0ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:55.579857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612e14b-b1b5-44d1-afbe-474bc91a0ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:55.580211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612e14b-b1b5-44d1-afbe-474bc91a0ed7 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:55.580977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612e14b-b1b5-44d1-afbe-474bc91a0ed7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:55.581369 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612e14b-b1b5-44d1-afbe-474bc91a0ed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:55.582957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612e14b-b1b5-44d1-afbe-474bc91a0ed7 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:55.594393 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 203/414] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:55.601079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9bcddc-a7ca-4541-9cc8-3cfc2c49430d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:55.733884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612e14b-b1b5-44d1-afbe-474bc91a0ed7 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:55.763586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:55.807014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9bcddc-a7ca-4541-9cc8-3cfc2c49430d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:55.848904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bec7e-74c5-4b12-84d4-be558c6cad40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:55.849539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bec7e-74c5-4b12-84d4-be558c6cad40 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:55.849824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bec7e-74c5-4b12-84d4-be558c6cad40 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:55.850099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bec7e-74c5-4b12-84d4-be558c6cad40 None admin] PATH_INFO: `/v3/groups/2079eaf7ffbc43eeb787b5f65841a2e6` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:55.851419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bec7e-74c5-4b12-84d4-be558c6cad40 None admin] RBAC: Authorizing `identity:get_group(group_id=2079eaf7ffbc43eeb787b5f65841a2e6)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:55.852912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bec7e-74c5-4b12-84d4-be558c6cad40 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:55.854902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 212/415] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:02:55 2022] GET /identity/v3/groups/2079eaf7ffbc43eeb787b5f65841a2e6 => generated 229 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:55.861811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bec7e-74c5-4b12-84d4-be558c6cad40 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:55.878983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612e14b-b1b5-44d1-afbe-474bc91a0ed7 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:55.910902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:55.956922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d6d0e-a614-4328-9ca9-cf285ed3b417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:55.957919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d6d0e-a614-4328-9ca9-cf285ed3b417 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:55.958423 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d6d0e-a614-4328-9ca9-cf285ed3b417 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:55.958965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d6d0e-a614-4328-9ca9-cf285ed3b417 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:55.980276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d6d0e-a614-4328-9ca9-cf285ed3b417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-d34d6d0e-a614-4328-9ca9-cf285ed3b417 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:55.993633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:55.994631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:55.995463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 204/416] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:55.997166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bec7e-74c5-4b12-84d4-be558c6cad40 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:56.003690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d6d0e-a614-4328-9ca9-cf285ed3b417 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:56.089500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640b649-069b-407c-8b90-4e89f1fa6e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:56.090703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640b649-069b-407c-8b90-4e89f1fa6e29 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:56.091056 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640b649-069b-407c-8b90-4e89f1fa6e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:56.091442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640b649-069b-407c-8b90-4e89f1fa6e29 None admin] PATH_INFO: `/v3/projects/236d122dbb6b4283a260bcccbdcd4a4a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:56.094894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640b649-069b-407c-8b90-4e89f1fa6e29 None admin] RBAC: Authorizing `identity:get_project(project_id=236d122dbb6b4283a260bcccbdcd4a4a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:56.100054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640b649-069b-407c-8b90-4e89f1fa6e29 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:56.102264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 213/417] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:55 2022] GET /identity/v3/projects/236d122dbb6b4283a260bcccbdcd4a4a => generated 344 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:56.112872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640b649-069b-407c-8b90-4e89f1fa6e29 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:56.148624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:56.176663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d6d0e-a614-4328-9ca9-cf285ed3b417 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:56.222045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ddf1e-ca50-4a2a-a332-5add888ecb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:56.223071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ddf1e-ca50-4a2a-a332-5add888ecb48 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:56.223734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ddf1e-ca50-4a2a-a332-5add888ecb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:56.224397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ddf1e-ca50-4a2a-a332-5add888ecb48 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:56.225321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ddf1e-ca50-4a2a-a332-5add888ecb48 None admin] RBAC: Adding query filter params (name=member) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:56.225826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ddf1e-ca50-4a2a-a332-5add888ecb48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:56.227789 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ddf1e-ca50-4a2a-a332-5add888ecb48 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:56.240090 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 205/418] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:02:56.245950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ddf1e-ca50-4a2a-a332-5add888ecb48 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:56.404264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640b649-069b-407c-8b90-4e89f1fa6e29 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:56.410727 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:56.485775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7a9cd-9a78-4851-a606-4619474d9cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:56.488251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7a9cd-9a78-4851-a606-4619474d9cd0 None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:56.489432 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7a9cd-9a78-4851-a606-4619474d9cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:56.489637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ddf1e-ca50-4a2a-a332-5add888ecb48 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:56.492077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7a9cd-9a78-4851-a606-4619474d9cd0 None admin] PATH_INFO: `/v3/projects/236d122dbb6b4283a260bcccbdcd4a4a/groups/2079eaf7ffbc43eeb787b5f65841a2e6/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:56.494924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7a9cd-9a78-4851-a606-4619474d9cd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=236d122dbb6b4283a260bcccbdcd4a4a, group_id=2079eaf7ffbc43eeb787b5f65841a2e6, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:56.497889 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7a9cd-9a78-4851-a606-4619474d9cd0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:56.516101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 214/419] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Aug 30 01:02:56 2022] PUT /identity/v3/projects/236d122dbb6b4283a260bcccbdcd4a4a/groups/2079eaf7ffbc43eeb787b5f65841a2e6/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:02:56.613259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c6869-7e4c-4cd7-bd00-d95575ce600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:56.614464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c6869-7e4c-4cd7-bd00-d95575ce600e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:56.614933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c6869-7e4c-4cd7-bd00-d95575ce600e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:56.615320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c6869-7e4c-4cd7-bd00-d95575ce600e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:56.633489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:56.779114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c6869-7e4c-4cd7-bd00-d95575ce600e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:56.783135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c6869-7e4c-4cd7-bd00-d95575ce600e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:56.952679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 206/420] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:56.961845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7a9cd-9a78-4851-a606-4619474d9cd0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:57.183083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7a9cd-9a78-4851-a606-4619474d9cd0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:57.277379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1842b2-b324-42db-a697-7fb863550715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:57.278913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1842b2-b324-42db-a697-7fb863550715 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.279665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1842b2-b324-42db-a697-7fb863550715 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.280401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1842b2-b324-42db-a697-7fb863550715 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.283061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1842b2-b324-42db-a697-7fb863550715 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:57.286155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1842b2-b324-42db-a697-7fb863550715 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:57.289605 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 215/421] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:56 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:57.295503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c6869-7e4c-4cd7-bd00-d95575ce600e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:57.387027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:57.509059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c6869-7e4c-4cd7-bd00-d95575ce600e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:57.591637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93ff18-f2fc-4465-a0a8-31095097f45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:57.592586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93ff18-f2fc-4465-a0a8-31095097f45f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.592932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93ff18-f2fc-4465-a0a8-31095097f45f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.593247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93ff18-f2fc-4465-a0a8-31095097f45f None admin] PATH_INFO: `/v3/groups/d90461ea6cfc4d41837a8a536af051b4` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.601135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93ff18-f2fc-4465-a0a8-31095097f45f None admin] RBAC: Authorizing `identity:get_group(group_id=d90461ea6cfc4d41837a8a536af051b4)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:57.603303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93ff18-f2fc-4465-a0a8-31095097f45f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:57.606469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 207/422] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:02:57 2022] GET /identity/v3/groups/d90461ea6cfc4d41837a8a536af051b4 => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:57.612340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1842b2-b324-42db-a697-7fb863550715 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:57.676908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:57.677685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd0f95-cae1-4540-87ca-b683d561fda9 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.678169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd0f95-cae1-4540-87ca-b683d561fda9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.678634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd0f95-cae1-4540-87ca-b683d561fda9 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.682602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 208/423] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:57.689379 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba9a06-3e5c-44f3-962c-14d589d8ad97 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.689812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba9a06-3e5c-44f3-962c-14d589d8ad97 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.690188 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba9a06-3e5c-44f3-962c-14d589d8ad97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.723845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-21ba9a06-3e5c-44f3-962c-14d589d8ad97 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:57.725856 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-21ba9a06-3e5c-44f3-962c-14d589d8ad97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:57.823750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 209/424] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:02:57.866840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1842b2-b324-42db-a697-7fb863550715 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:57.928116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95d9e0-9e83-4731-8347-6ee62ef024a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:57.929144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95d9e0-9e83-4731-8347-6ee62ef024a9 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:57.929620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95d9e0-9e83-4731-8347-6ee62ef024a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:57.930104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95d9e0-9e83-4731-8347-6ee62ef024a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:57.931400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95d9e0-9e83-4731-8347-6ee62ef024a9 None admin] RBAC: Adding query filter params (group.id=d90461ea6cfc4d41837a8a536af051b4, role.id=b07ea738cc204b2b8125bf4cd6d9c54a, scope.project.id=8a6d5bbd6e874220b0d317fa4794df82) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:57.931951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95d9e0-9e83-4731-8347-6ee62ef024a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:57.934686 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95d9e0-9e83-4731-8347-6ee62ef024a9 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:57.944884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 216/425] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Aug 30 01:02:57 2022] GET /identity/v3/role_assignments?group.id=d90461ea6cfc4d41837a8a536af051b4&scope.project.id=8a6d5bbd6e874220b0d317fa4794df82&role.id=b07ea738cc204b2b8125bf4cd6d9c54a => generated 611 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:58.092549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd41b0-ca36-406c-a987-1c04d40a7853 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:58.092855 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd41b0-ca36-406c-a987-1c04d40a7853 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:58.093200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd41b0-ca36-406c-a987-1c04d40a7853 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:58.095225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 210/426] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:58.101430 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db0a2d-fc95-4620-b513-38a33aed744c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:58.102088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db0a2d-fc95-4620-b513-38a33aed744c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:58.102556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db0a2d-fc95-4620-b513-38a33aed744c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:58.127385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-81db0a2d-fc95-4620-b513-38a33aed744c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:58.129073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-81db0a2d-fc95-4620-b513-38a33aed744c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:58.226597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 217/427] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:58.290581 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd41b0-ca36-406c-a987-1c04d40a7853 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:58.346086 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:58.402025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea959895-6774-46eb-8153-b2564ea0d34d None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:58.402388 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea959895-6774-46eb-8153-b2564ea0d34d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:58.402573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea959895-6774-46eb-8153-b2564ea0d34d None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:58.404671 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 218/428] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:58.411552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1218dc-71da-44a9-b689-6bab53620b40 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:58.411737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1218dc-71da-44a9-b689-6bab53620b40 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:58.412007 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1218dc-71da-44a9-b689-6bab53620b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:58.442390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-1a1218dc-71da-44a9-b689-6bab53620b40 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:58.443644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1a1218dc-71da-44a9-b689-6bab53620b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:58.464742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd41b0-ca36-406c-a987-1c04d40a7853 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:58.478024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:58.519171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378aa870-2a0c-4352-9dae-80485fe86245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:58.519912 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378aa870-2a0c-4352-9dae-80485fe86245 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:58.520184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378aa870-2a0c-4352-9dae-80485fe86245 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:58.520450 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378aa870-2a0c-4352-9dae-80485fe86245 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:58.532912 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378aa870-2a0c-4352-9dae-80485fe86245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-378aa870-2a0c-4352-9dae-80485fe86245 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:58.538817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:58.539840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:02:58.540720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 211/429] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:02:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:58.546478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 219/430] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:58.553137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378aa870-2a0c-4352-9dae-80485fe86245 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:58.600326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1218dc-71da-44a9-b689-6bab53620b40 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:58.700084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:58.709734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378aa870-2a0c-4352-9dae-80485fe86245 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:58.754061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1218dc-71da-44a9-b689-6bab53620b40 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:58.765800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f766e78-b7ed-4985-ba0f-4d65233fee04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:58.767462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f766e78-b7ed-4985-ba0f-4d65233fee04 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:58.768054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f766e78-b7ed-4985-ba0f-4d65233fee04 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:58.768680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f766e78-b7ed-4985-ba0f-4d65233fee04 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:58.769811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f766e78-b7ed-4985-ba0f-4d65233fee04 None admin] RBAC: Adding query filter params (name=member) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:58.770361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f766e78-b7ed-4985-ba0f-4d65233fee04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:58.772439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f766e78-b7ed-4985-ba0f-4d65233fee04 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:58.789924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 212/431] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:02:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:58.792247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f766e78-b7ed-4985-ba0f-4d65233fee04 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:58.796430 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba546-823c-42f4-8923-bb20edc1cac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:58.798475 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba546-823c-42f4-8923-bb20edc1cac2 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:58.798475 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba546-823c-42f4-8923-bb20edc1cac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:58.798475 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba546-823c-42f4-8923-bb20edc1cac2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:58.822888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ba546-823c-42f4-8923-bb20edc1cac2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-069ba546-823c-42f4-8923-bb20edc1cac2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:58.828728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:58.830370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:58.831177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:58.831177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:58.831177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:58.831177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:58.831177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:58.831177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:58.831177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:58.831177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:58.831177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:58.831177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:58.831177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:58.831177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:58.833907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 220/432] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:02:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:58.839575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba546-823c-42f4-8923-bb20edc1cac2 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:58.863486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:58.977045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f766e78-b7ed-4985-ba0f-4d65233fee04 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:58.980752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba546-823c-42f4-8923-bb20edc1cac2 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:59.022432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac06e20-04c8-44c1-beb3-87e6a164e474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.023047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac06e20-04c8-44c1-beb3-87e6a164e474 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.023331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac06e20-04c8-44c1-beb3-87e6a164e474 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.023606 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac06e20-04c8-44c1-beb3-87e6a164e474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.024843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222e548-6282-4694-8a91-45c598fcaeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.025522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222e548-6282-4694-8a91-45c598fcaeb1 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.025807 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222e548-6282-4694-8a91-45c598fcaeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.026075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222e548-6282-4694-8a91-45c598fcaeb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.026758 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222e548-6282-4694-8a91-45c598fcaeb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:59.028024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222e548-6282-4694-8a91-45c598fcaeb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:59.028969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222e548-6282-4694-8a91-45c598fcaeb1 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:59.045665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 221/433] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:02:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:59.045665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222e548-6282-4694-8a91-45c598fcaeb1 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:59.061543 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:59.121871 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac06e20-04c8-44c1-beb3-87e6a164e474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:59.123246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac06e20-04c8-44c1-beb3-87e6a164e474 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:59.146267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:59.173001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222e548-6282-4694-8a91-45c598fcaeb1 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:59.216786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cb7f5-8f28-4b73-9f35-05d941bd3b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.217457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cb7f5-8f28-4b73-9f35-05d941bd3b30 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.217745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cb7f5-8f28-4b73-9f35-05d941bd3b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.218041 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cb7f5-8f28-4b73-9f35-05d941bd3b30 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.229532 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cb7f5-8f28-4b73-9f35-05d941bd3b30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-d53cb7f5-8f28-4b73-9f35-05d941bd3b30 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:02:59.235520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:02:59.236618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:02:59.237418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:02:59.237418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:02:59.237418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:02:59.237418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:02:59.237418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:02:59.237418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:02:59.237418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:02:59.237418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:02:59.237418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:02:59.237418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:02:59.237418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:02:59.237418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:02:59.238010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 213/434] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:59.238010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac06e20-04c8-44c1-beb3-87e6a164e474 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:59.238139 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 222/435] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:02:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:02:59.239072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb498da-6525-4c9c-9555-00db728949fb None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.239352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb498da-6525-4c9c-9555-00db728949fb None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.239623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb498da-6525-4c9c-9555-00db728949fb None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.242991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 223/436] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:59.242991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb498da-6525-4c9c-9555-00db728949fb None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:59.305368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:59.353891 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac06e20-04c8-44c1-beb3-87e6a164e474 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:59.362554 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb498da-6525-4c9c-9555-00db728949fb None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:59.395552 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20208e9c-29e0-4f7d-a80e-0b9f419d68f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.396318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20208e9c-29e0-4f7d-a80e-0b9f419d68f2 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.396595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20208e9c-29e0-4f7d-a80e-0b9f419d68f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.396922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20208e9c-29e0-4f7d-a80e-0b9f419d68f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.402959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b6aa9-9132-49fc-93d8-47847f2e84ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.403601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b6aa9-9132-49fc-93d8-47847f2e84ac None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.403880 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b6aa9-9132-49fc-93d8-47847f2e84ac None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.404193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b6aa9-9132-49fc-93d8-47847f2e84ac None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.405366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b6aa9-9132-49fc-93d8-47847f2e84ac None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:59.407423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b6aa9-9132-49fc-93d8-47847f2e84ac None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:59.409259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 224/437] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:59 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:59.410136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b6aa9-9132-49fc-93d8-47847f2e84ac None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:59.419936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:59.479848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20208e9c-29e0-4f7d-a80e-0b9f419d68f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:59.481548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20208e9c-29e0-4f7d-a80e-0b9f419d68f2 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:59.526722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b6aa9-9132-49fc-93d8-47847f2e84ac None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:59.553976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:59.580896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc15fd1-d805-4210-804f-66e58a522102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.581905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc15fd1-d805-4210-804f-66e58a522102 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.582337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc15fd1-d805-4210-804f-66e58a522102 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.582606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc15fd1-d805-4210-804f-66e58a522102 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.583281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc15fd1-d805-4210-804f-66e58a522102 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:02:59.583617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc15fd1-d805-4210-804f-66e58a522102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:59.585168 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc15fd1-d805-4210-804f-66e58a522102 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:59.594107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 225/438] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:02:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:59.595293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364829a-adf8-41c7-a90f-ed5d6c6ccb2c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.595592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364829a-adf8-41c7-a90f-ed5d6c6ccb2c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.595861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364829a-adf8-41c7-a90f-ed5d6c6ccb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.608031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 214/439] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:02:59.609402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20208e9c-29e0-4f7d-a80e-0b9f419d68f2 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:59.620291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-2364829a-adf8-41c7-a90f-ed5d6c6ccb2c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:59.621474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2364829a-adf8-41c7-a90f-ed5d6c6ccb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:59.694327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:59.700941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 226/440] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:02:59.702064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364829a-adf8-41c7-a90f-ed5d6c6ccb2c None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:59.736744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20208e9c-29e0-4f7d-a80e-0b9f419d68f2 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:59.786082 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b881b-3ca2-424a-a1d2-f7395de52f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.786825 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b881b-3ca2-424a-a1d2-f7395de52f11 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.787185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b881b-3ca2-424a-a1d2-f7395de52f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.787474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b881b-3ca2-424a-a1d2-f7395de52f11 None admin] PATH_INFO: `/v3/groups/d90461ea6cfc4d41837a8a536af051b4` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.788688 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b881b-3ca2-424a-a1d2-f7395de52f11 None admin] RBAC: Authorizing `identity:get_group(group_id=d90461ea6cfc4d41837a8a536af051b4)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:59.790396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b881b-3ca2-424a-a1d2-f7395de52f11 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:59.792267 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 215/441] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:02:59 2022] GET /identity/v3/groups/d90461ea6cfc4d41837a8a536af051b4 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:59.793119 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b881b-3ca2-424a-a1d2-f7395de52f11 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:59.830861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364829a-adf8-41c7-a90f-ed5d6c6ccb2c None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:59.831753 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:59.872906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103f9e3-a196-4dfb-a8ec-dbce6e2dedb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.873944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103f9e3-a196-4dfb-a8ec-dbce6e2dedb1 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.874551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103f9e3-a196-4dfb-a8ec-dbce6e2dedb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.874830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103f9e3-a196-4dfb-a8ec-dbce6e2dedb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.917781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b881b-3ca2-424a-a1d2-f7395de52f11 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:59.953284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:59.958280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103f9e3-a196-4dfb-a8ec-dbce6e2dedb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:59.960548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103f9e3-a196-4dfb-a8ec-dbce6e2dedb1 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:59.962741 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514a389-ed4a-4bbf-a110-50fdf770e8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.963672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514a389-ed4a-4bbf-a110-50fdf770e8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.964942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514a389-ed4a-4bbf-a110-50fdf770e8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.965274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514a389-ed4a-4bbf-a110-50fdf770e8c6 None admin] PATH_INFO: `/v3/projects/8a6d5bbd6e874220b0d317fa4794df82` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.966526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514a389-ed4a-4bbf-a110-50fdf770e8c6 None admin] RBAC: Authorizing `identity:get_project(project_id=8a6d5bbd6e874220b0d317fa4794df82)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:59.968163 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514a389-ed4a-4bbf-a110-50fdf770e8c6 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:59.976153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 216/442] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:02:59 2022] GET /identity/v3/projects/8a6d5bbd6e874220b0d317fa4794df82 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:02:59.976153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514a389-ed4a-4bbf-a110-50fdf770e8c6 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.085984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:00.092382 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 227/443] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:02:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:03:00.093534 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103f9e3-a196-4dfb-a8ec-dbce6e2dedb1 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.095554 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514a389-ed4a-4bbf-a110-50fdf770e8c6 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.144804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7304198-6094-4b05-8b6c-b11b5d53793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.145414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7304198-6094-4b05-8b6c-b11b5d53793a None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.145690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7304198-6094-4b05-8b6c-b11b5d53793a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.145959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7304198-6094-4b05-8b6c-b11b5d53793a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.146712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7304198-6094-4b05-8b6c-b11b5d53793a None admin] RBAC: Adding query filter params (group.id=d90461ea6cfc4d41837a8a536af051b4, role.id=b07ea738cc204b2b8125bf4cd6d9c54a, scope.project.id=dcbc59eca63d4c3ea02d6e944948c888) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:00.147023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7304198-6094-4b05-8b6c-b11b5d53793a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:00.148615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7304198-6094-4b05-8b6c-b11b5d53793a None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:00.155371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 217/444] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Aug 30 01:02:59 2022] GET /identity/v3/role_assignments?group.id=d90461ea6cfc4d41837a8a536af051b4&scope.project.id=dcbc59eca63d4c3ea02d6e944948c888&role.id=b07ea738cc204b2b8125bf4cd6d9c54a => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:00.156280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7304198-6094-4b05-8b6c-b11b5d53793a None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.240251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:00.245330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103f9e3-a196-4dfb-a8ec-dbce6e2dedb1 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.299135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d6e55-2caf-4acd-8887-05340c1d3ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.300411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d6e55-2caf-4acd-8887-05340c1d3ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.300867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d6e55-2caf-4acd-8887-05340c1d3ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.301224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d6e55-2caf-4acd-8887-05340c1d3ff8 None admin] PATH_INFO: `/v3/groups/d90461ea6cfc4d41837a8a536af051b4` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.303030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d6e55-2caf-4acd-8887-05340c1d3ff8 None admin] RBAC: Authorizing `identity:get_group(group_id=d90461ea6cfc4d41837a8a536af051b4)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:00.305286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d6e55-2caf-4acd-8887-05340c1d3ff8 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:00.307693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 228/445] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:03:00 2022] GET /identity/v3/groups/d90461ea6cfc4d41837a8a536af051b4 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:00.308830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d6e55-2caf-4acd-8887-05340c1d3ff8 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.332027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7304198-6094-4b05-8b6c-b11b5d53793a None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.380292 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25a45d-8b1c-42d4-886a-a79683277846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.380932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25a45d-8b1c-42d4-886a-a79683277846 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.381259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25a45d-8b1c-42d4-886a-a79683277846 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.381476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25a45d-8b1c-42d4-886a-a79683277846 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.393924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25a45d-8b1c-42d4-886a-a79683277846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-bf25a45d-8b1c-42d4-886a-a79683277846 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:00.399456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:00.400644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 218/446] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:00.402296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25a45d-8b1c-42d4-886a-a79683277846 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.424675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:00.444383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d6e55-2caf-4acd-8887-05340c1d3ff8 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.482365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0c202-d22e-4e39-8390-5dec647fb6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.483063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0c202-d22e-4e39-8390-5dec647fb6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.483312 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0c202-d22e-4e39-8390-5dec647fb6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.483585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0c202-d22e-4e39-8390-5dec647fb6e3 None admin] PATH_INFO: `/v3/projects/dcbc59eca63d4c3ea02d6e944948c888` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.484807 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0c202-d22e-4e39-8390-5dec647fb6e3 None admin] RBAC: Authorizing `identity:get_project(project_id=dcbc59eca63d4c3ea02d6e944948c888)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:00.486867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0c202-d22e-4e39-8390-5dec647fb6e3 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:00.488668 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 229/447] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:03:00 2022] GET /identity/v3/projects/dcbc59eca63d4c3ea02d6e944948c888 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:00.489470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0c202-d22e-4e39-8390-5dec647fb6e3 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.527918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25a45d-8b1c-42d4-886a-a79683277846 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.528615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:00.570810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3047f-ee86-4517-abd5-8bd289a7deae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.571547 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3047f-ee86-4517-abd5-8bd289a7deae None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.571689 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3047f-ee86-4517-abd5-8bd289a7deae None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.572034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3047f-ee86-4517-abd5-8bd289a7deae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.572714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3047f-ee86-4517-abd5-8bd289a7deae None admin] RBAC: Adding query filter params (group.id=d90461ea6cfc4d41837a8a536af051b4, role.id=7562948bd63d483cad68427819d434a9, scope.project.id=8a6d5bbd6e874220b0d317fa4794df82) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:00.573016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3047f-ee86-4517-abd5-8bd289a7deae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:00.574624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3047f-ee86-4517-abd5-8bd289a7deae None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:00.581221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 219/448] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Aug 30 01:03:00 2022] GET /identity/v3/role_assignments?group.id=d90461ea6cfc4d41837a8a536af051b4&scope.project.id=8a6d5bbd6e874220b0d317fa4794df82&role.id=7562948bd63d483cad68427819d434a9 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:00.582143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3047f-ee86-4517-abd5-8bd289a7deae None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.621003 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0c202-d22e-4e39-8390-5dec647fb6e3 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.671499 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63e68d-e43d-406a-ba13-634f3b921ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.672168 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63e68d-e43d-406a-ba13-634f3b921ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.673006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63e68d-e43d-406a-ba13-634f3b921ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.673336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63e68d-e43d-406a-ba13-634f3b921ef8 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.674128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63e68d-e43d-406a-ba13-634f3b921ef8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:00.674583 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63e68d-e43d-406a-ba13-634f3b921ef8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:00.676026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63e68d-e43d-406a-ba13-634f3b921ef8 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:00.692338 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 230/449] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:00.699145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63e68d-e43d-406a-ba13-634f3b921ef8 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.715250 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:00.738618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3047f-ee86-4517-abd5-8bd289a7deae None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.788248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981238-b836-4ab6-a797-06232913d9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.788831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981238-b836-4ab6-a797-06232913d9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.789108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981238-b836-4ab6-a797-06232913d9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.789379 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981238-b836-4ab6-a797-06232913d9b0 None admin] PATH_INFO: `/v3/groups/d90461ea6cfc4d41837a8a536af051b4` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.790555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee981238-b836-4ab6-a797-06232913d9b0 None admin] RBAC: Authorizing `identity:get_group(group_id=d90461ea6cfc4d41837a8a536af051b4)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:00.792065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee981238-b836-4ab6-a797-06232913d9b0 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:00.793842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 220/450] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:03:00 2022] GET /identity/v3/groups/d90461ea6cfc4d41837a8a536af051b4 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:00.802157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981238-b836-4ab6-a797-06232913d9b0 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:00.857842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:00.865791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63e68d-e43d-406a-ba13-634f3b921ef8 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:00.910898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d9936-e1ae-4f49-bbd4-564964cac618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:00.912392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d9936-e1ae-4f49-bbd4-564964cac618 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:00.912719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d9936-e1ae-4f49-bbd4-564964cac618 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:00.913040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d9936-e1ae-4f49-bbd4-564964cac618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:00.954888 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981238-b836-4ab6-a797-06232913d9b0 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.011868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655c596-7400-48dc-9af1-6426c06e1334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.013494 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655c596-7400-48dc-9af1-6426c06e1334 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.013865 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655c596-7400-48dc-9af1-6426c06e1334 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.014251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655c596-7400-48dc-9af1-6426c06e1334 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.015240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655c596-7400-48dc-9af1-6426c06e1334 None admin] RBAC: Adding query filter params (group.id=d90461ea6cfc4d41837a8a536af051b4, role.id=7562948bd63d483cad68427819d434a9, scope.project.id=dcbc59eca63d4c3ea02d6e944948c888) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:01.015749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655c596-7400-48dc-9af1-6426c06e1334 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655c596-7400-48dc-9af1-6426c06e1334 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.022400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d9936-e1ae-4f49-bbd4-564964cac618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.023987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d9936-e1ae-4f49-bbd4-564964cac618 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.030471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 221/451] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Aug 30 01:03:00 2022] GET /identity/v3/role_assignments?group.id=d90461ea6cfc4d41837a8a536af051b4&scope.project.id=dcbc59eca63d4c3ea02d6e944948c888&role.id=7562948bd63d483cad68427819d434a9 => generated 611 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:01.185058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:01.194386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 231/452] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 01:03:01.200850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655c596-7400-48dc-9af1-6426c06e1334 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.223124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:01.324753 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655c596-7400-48dc-9af1-6426c06e1334 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.384396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf0ccf-e44f-4d00-860a-ee37cb2fd01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.385130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf0ccf-e44f-4d00-860a-ee37cb2fd01e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.385519 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf0ccf-e44f-4d00-860a-ee37cb2fd01e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.385892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf0ccf-e44f-4d00-860a-ee37cb2fd01e None admin] PATH_INFO: `/v3/projects/236d122dbb6b4283a260bcccbdcd4a4a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.387169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf0ccf-e44f-4d00-860a-ee37cb2fd01e None admin] RBAC: Authorizing `identity:get_project(project_id=236d122dbb6b4283a260bcccbdcd4a4a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.388676 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf0ccf-e44f-4d00-860a-ee37cb2fd01e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.390350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 222/453] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:03:01 2022] GET /identity/v3/projects/236d122dbb6b4283a260bcccbdcd4a4a => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:01.397297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d9936-e1ae-4f49-bbd4-564964cac618 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.506287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d9936-e1ae-4f49-bbd4-564964cac618 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.550714 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429264a9-72bf-40ba-9174-4c22b5aa4372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.551590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429264a9-72bf-40ba-9174-4c22b5aa4372 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.551898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429264a9-72bf-40ba-9174-4c22b5aa4372 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.552296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429264a9-72bf-40ba-9174-4c22b5aa4372 None admin] PATH_INFO: `/v3/groups/2079eaf7ffbc43eeb787b5f65841a2e6` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.553694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429264a9-72bf-40ba-9174-4c22b5aa4372 None admin] RBAC: Authorizing `identity:get_group(group_id=2079eaf7ffbc43eeb787b5f65841a2e6)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.555431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429264a9-72bf-40ba-9174-4c22b5aa4372 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.557205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 232/454] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Aug 30 01:03:01 2022] GET /identity/v3/groups/2079eaf7ffbc43eeb787b5f65841a2e6 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:01.564248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf0ccf-e44f-4d00-860a-ee37cb2fd01e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:01.652482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:01.674945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf0ccf-e44f-4d00-860a-ee37cb2fd01e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:01.714007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f87d0-d15c-4ac9-a95c-26f6d8b66e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:01.714618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f87d0-d15c-4ac9-a95c-26f6d8b66e37 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:01.714897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f87d0-d15c-4ac9-a95c-26f6d8b66e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:01.715167 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f87d0-d15c-4ac9-a95c-26f6d8b66e37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:01.715905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f87d0-d15c-4ac9-a95c-26f6d8b66e37 None admin] RBAC: Adding query filter params (group.id=2079eaf7ffbc43eeb787b5f65841a2e6, role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=236d122dbb6b4283a260bcccbdcd4a4a) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:01.716248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f87d0-d15c-4ac9-a95c-26f6d8b66e37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:01.717695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f87d0-d15c-4ac9-a95c-26f6d8b66e37 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:01.724624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 223/455] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Aug 30 01:03:01 2022] GET /identity/v3/role_assignments?group.id=2079eaf7ffbc43eeb787b5f65841a2e6&scope.project.id=236d122dbb6b4283a260bcccbdcd4a4a&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:04.385017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a173e7e-ee9e-44b8-b5f3-03fd8261fee7 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:04.386245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a173e7e-ee9e-44b8-b5f3-03fd8261fee7 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:04.386779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a173e7e-ee9e-44b8-b5f3-03fd8261fee7 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:04.389016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 233/456] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:04.396212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bccc9-73da-44cb-a429-f74de347a697 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:04.397138 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bccc9-73da-44cb-a429-f74de347a697 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:04.397690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bccc9-73da-44cb-a429-f74de347a697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:04.425418 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-af4bccc9-73da-44cb-a429-f74de347a697 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:04.427100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-af4bccc9-73da-44cb-a429-f74de347a697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:04.530725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 224/457] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:04.732896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b0dc3-63e8-42ed-944f-98ada7f997b3 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:04.732896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b0dc3-63e8-42ed-944f-98ada7f997b3 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:04.732896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b0dc3-63e8-42ed-944f-98ada7f997b3 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:04.733675 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 234/458] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:04.739371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43274699-f9cb-40c1-95e8-a5742ac9b373 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:04.739651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43274699-f9cb-40c1-95e8-a5742ac9b373 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:04.739921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43274699-f9cb-40c1-95e8-a5742ac9b373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:04.773113 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-43274699-f9cb-40c1-95e8-a5742ac9b373 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:04.774320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-43274699-f9cb-40c1-95e8-a5742ac9b373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:04.845711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30260c86-e7ae-4295-8524-3fcc8676254c None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:04.846078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30260c86-e7ae-4295-8524-3fcc8676254c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:04.846426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30260c86-e7ae-4295-8524-3fcc8676254c None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:04.848183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 235/459] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:04.853791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ef94d-3eb1-4b82-b77b-27d7eab15582 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:04.854066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ef94d-3eb1-4b82-b77b-27d7eab15582 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:04.854341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ef94d-3eb1-4b82-b77b-27d7eab15582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:04.894647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 225/460] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:04.896745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-5d2ef94d-3eb1-4b82-b77b-27d7eab15582 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:04.898424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5d2ef94d-3eb1-4b82-b77b-27d7eab15582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:05.002116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:05.021585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 236/461] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:03:05.076532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43274699-f9cb-40c1-95e8-a5742ac9b373 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:05.087587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:05.221657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43274699-f9cb-40c1-95e8-a5742ac9b373 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:05.269093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfeaed9-a6f9-4d01-a4dc-b2a7024ceee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:05.269679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfeaed9-a6f9-4d01-a4dc-b2a7024ceee0 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:05.269972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfeaed9-a6f9-4d01-a4dc-b2a7024ceee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:05.270316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfeaed9-a6f9-4d01-a4dc-b2a7024ceee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:05.271386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ef94d-3eb1-4b82-b77b-27d7eab15582 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:05.286100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfeaed9-a6f9-4d01-a4dc-b2a7024ceee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:05.288166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfeaed9-a6f9-4d01-a4dc-b2a7024ceee0 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-9cfeaed9-a6f9-4d01-a4dc-b2a7024ceee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:05.309536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:05.310516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:05.311364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:05.311364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:05.311364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:05.311364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 226/462] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:05.317383 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfeaed9-a6f9-4d01-a4dc-b2a7024ceee0 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:05.400330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:05.498440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ef94d-3eb1-4b82-b77b-27d7eab15582 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:05.502250 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfeaed9-a6f9-4d01-a4dc-b2a7024ceee0 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:05.551259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0532b4-34ec-42cb-928d-666f3e7f0e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:05.552711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0532b4-34ec-42cb-928d-666f3e7f0e02 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:05.553778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0532b4-34ec-42cb-928d-666f3e7f0e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:05.554301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0532b4-34ec-42cb-928d-666f3e7f0e02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:05.567409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634a323-fccd-440c-99ea-7c7e3c68aa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:05.569018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634a323-fccd-440c-99ea-7c7e3c68aa2b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:05.569775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634a323-fccd-440c-99ea-7c7e3c68aa2b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:05.570310 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0532b4-34ec-42cb-928d-666f3e7f0e02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:05.571004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634a323-fccd-440c-99ea-7c7e3c68aa2b None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:05.572299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0532b4-34ec-42cb-928d-666f3e7f0e02 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:05.572893 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634a323-fccd-440c-99ea-7c7e3c68aa2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:05.573704 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634a323-fccd-440c-99ea-7c7e3c68aa2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:05.576841 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634a323-fccd-440c-99ea-7c7e3c68aa2b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-aa0532b4-34ec-42cb-928d-666f3e7f0e02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:05.595483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:05.596539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:05.597628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:05.597628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:05.597628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:05.597628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 237/463] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:05.598790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0532b4-34ec-42cb-928d-666f3e7f0e02 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:05.599710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 227/464] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:05.607881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632ac14-00fc-4f89-ae3a-18a2febce4c7 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:05.607881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632ac14-00fc-4f89-ae3a-18a2febce4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:05.607881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632ac14-00fc-4f89-ae3a-18a2febce4c7 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:05.607881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 228/465] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:05.612543 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632ac14-00fc-4f89-ae3a-18a2febce4c7 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:05.667110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:05.754173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632ac14-00fc-4f89-ae3a-18a2febce4c7 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:05.758739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0532b4-34ec-42cb-928d-666f3e7f0e02 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:05.775026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:05.800417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db970f90-3cbe-4f03-8cd2-87320216eb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:05.801130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db970f90-3cbe-4f03-8cd2-87320216eb18 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:05.801570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db970f90-3cbe-4f03-8cd2-87320216eb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:05.801782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db970f90-3cbe-4f03-8cd2-87320216eb18 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:05.802697 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db970f90-3cbe-4f03-8cd2-87320216eb18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:05.803146 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db970f90-3cbe-4f03-8cd2-87320216eb18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:05.803306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b650ace-5bbc-40c3-a9db-7a44602b3fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:05.804395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b650ace-5bbc-40c3-a9db-7a44602b3fcb None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:05.804881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b650ace-5bbc-40c3-a9db-7a44602b3fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:05.805016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db970f90-3cbe-4f03-8cd2-87320216eb18 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:05.805360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b650ace-5bbc-40c3-a9db-7a44602b3fcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:05.824029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 229/466] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:05.824029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db970f90-3cbe-4f03-8cd2-87320216eb18 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:05.827594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b650ace-5bbc-40c3-a9db-7a44602b3fcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:05.830304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b650ace-5bbc-40c3-a9db-7a44602b3fcb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-9b650ace-5bbc-40c3-a9db-7a44602b3fcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:05.848952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:05.849970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:05.850966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:05.850966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:05.850966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:05.850966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 238/467] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:05.851817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d08c72-146f-44b3-936f-1e83f6861023 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:05.851817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d08c72-146f-44b3-936f-1e83f6861023 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:05.851817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d08c72-146f-44b3-936f-1e83f6861023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:05.873617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-41d08c72-146f-44b3-936f-1e83f6861023 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:05.876469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-41d08c72-146f-44b3-936f-1e83f6861023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:05.984196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db970f90-3cbe-4f03-8cd2-87320216eb18 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:06.002988 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 239/468] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:06.002988 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d08c72-146f-44b3-936f-1e83f6861023 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:06.048035 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:06.050108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ad91d-1e85-4737-be4c-2814d5734e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:06.051008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ad91d-1e85-4737-be4c-2814d5734e89 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:06.051758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ad91d-1e85-4737-be4c-2814d5734e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:06.052700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ad91d-1e85-4737-be4c-2814d5734e89 None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:06.053957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ad91d-1e85-4737-be4c-2814d5734e89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:06.055787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ad91d-1e85-4737-be4c-2814d5734e89 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:06.086018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 230/469] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:03:05 2022] POST /identity/v3/users => generated 246 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:06.087020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ad91d-1e85-4737-be4c-2814d5734e89 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:06.144257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:06.161619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d08c72-146f-44b3-936f-1e83f6861023 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:06.237885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca61dc-a08b-4dd0-9634-a89a804f8e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:06.238570 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca61dc-a08b-4dd0-9634-a89a804f8e5a None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:06.238848 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca61dc-a08b-4dd0-9634-a89a804f8e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:06.239119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca61dc-a08b-4dd0-9634-a89a804f8e5a None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:06.242951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca61dc-a08b-4dd0-9634-a89a804f8e5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:06.244467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca61dc-a08b-4dd0-9634-a89a804f8e5a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:06.262904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ad91d-1e85-4737-be4c-2814d5734e89 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:06.282708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 240/470] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:03:05 2022] POST /identity/v3/users => generated 248 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:06.326583 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd443bc-31c8-4aa9-b739-be17c90caef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:06.327420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd443bc-31c8-4aa9-b739-be17c90caef1 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:06.327720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd443bc-31c8-4aa9-b739-be17c90caef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:06.328045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd443bc-31c8-4aa9-b739-be17c90caef1 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:06.329005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd443bc-31c8-4aa9-b739-be17c90caef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:06.329574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd443bc-31c8-4aa9-b739-be17c90caef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:06.331712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd443bc-31c8-4aa9-b739-be17c90caef1 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:06.333510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72e3e-ec8e-4d47-aec0-d8c1b3112aee None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:06.333794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72e3e-ec8e-4d47-aec0-d8c1b3112aee None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:06.334065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72e3e-ec8e-4d47-aec0-d8c1b3112aee None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:06.335542 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 241/471] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:03:06.342063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65908696-4a27-45af-ad24-cbc92b73614c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:06.342348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65908696-4a27-45af-ad24-cbc92b73614c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:06.342711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65908696-4a27-45af-ad24-cbc92b73614c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:06.355269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 231/472] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:06.367079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd443bc-31c8-4aa9-b739-be17c90caef1 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:06.387981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-65908696-4a27-45af-ad24-cbc92b73614c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:06.389800 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-65908696-4a27-45af-ad24-cbc92b73614c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:06.511971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 242/473] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:06.513286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65908696-4a27-45af-ad24-cbc92b73614c None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:06.538971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:06.560488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:06.568837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd443bc-31c8-4aa9-b739-be17c90caef1 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:06.628926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a1396-9a54-4c9e-87b7-47b88e85a2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:06.629642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a1396-9a54-4c9e-87b7-47b88e85a2ec None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:06.630224 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a1396-9a54-4c9e-87b7-47b88e85a2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:06.630656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a1396-9a54-4c9e-87b7-47b88e85a2ec None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:06.631530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a1396-9a54-4c9e-87b7-47b88e85a2ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:06.634845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a1396-9a54-4c9e-87b7-47b88e85a2ec None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:06.676693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 232/474] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:03:06 2022] POST /identity/v3/users => generated 248 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:06.677827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65908696-4a27-45af-ad24-cbc92b73614c None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:06.771179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed278d-4efc-4364-8e65-8252bce084e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:06.771842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed278d-4efc-4364-8e65-8252bce084e8 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:06.772233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed278d-4efc-4364-8e65-8252bce084e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:06.772568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed278d-4efc-4364-8e65-8252bce084e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:06.786976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed278d-4efc-4364-8e65-8252bce084e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:06.788675 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed278d-4efc-4364-8e65-8252bce084e8 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-4fed278d-4efc-4364-8e65-8252bce084e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:06.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:06.809342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:06.810193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:06.810193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:06.810193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:06.815310 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 243/475] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:06.816839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a1396-9a54-4c9e-87b7-47b88e85a2ec None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:06.849634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed278d-4efc-4364-8e65-8252bce084e8 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:06.976097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:07.025181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a1396-9a54-4c9e-87b7-47b88e85a2ec None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:07.034063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed278d-4efc-4364-8e65-8252bce084e8 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:07.076344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb92e2-f3bf-4ddd-bdb3-79c69e4bd543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:07.077733 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb92e2-f3bf-4ddd-bdb3-79c69e4bd543 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:07.079645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb92e2-f3bf-4ddd-bdb3-79c69e4bd543 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:07.080444 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb92e2-f3bf-4ddd-bdb3-79c69e4bd543 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:07.081909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb92e2-f3bf-4ddd-bdb3-79c69e4bd543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:07.082880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb92e2-f3bf-4ddd-bdb3-79c69e4bd543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:07.085663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb92e2-f3bf-4ddd-bdb3-79c69e4bd543 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:07.102918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 233/476] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:07.109372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf22da3-8f67-4718-bacd-dea061f04a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:07.110253 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb92e2-f3bf-4ddd-bdb3-79c69e4bd543 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:07.110625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf22da3-8f67-4718-bacd-dea061f04a4b None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:07.110984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf22da3-8f67-4718-bacd-dea061f04a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:07.112088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf22da3-8f67-4718-bacd-dea061f04a4b None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:07.112603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf22da3-8f67-4718-bacd-dea061f04a4b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:07.114321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf22da3-8f67-4718-bacd-dea061f04a4b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:07.128455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.sql.core [None req-adf22da3-8f67-4718-bacd-dea061f04a4b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 01:03:07.128455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 01:03:07.128455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [parameters: {'id': 'af96f81fae014cd081bbde146fb13a15', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 01:03:07.128455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81647) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 01:03:07.131466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-adf22da3-8f67-4718-bacd-dea061f04a4b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application conn.query(q) Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result.read() Aug 30 01:03:07.164987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return ref.to_dict() Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 01:03:07.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application next(self.gen) Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application yield resource Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application next(self.gen) Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise self.value Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application session.commit() Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 01:03:07.166847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.session.flush() Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application compat.raise_( Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 01:03:07.167697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 01:03:07.169410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application conn.query(q) Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result.read() Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 01:03:07.171685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [parameters: {'id': 'af96f81fae014cd081bbde146fb13a15', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:07.173779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 01:03:07.176068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:07.178188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 244/477] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Aug 30 01:03:06 2022] POST /identity/v3/roles => generated 144 bytes in 318 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:07.178188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf22da3-8f67-4718-bacd-dea061f04a4b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:07.273626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb92e2-f3bf-4ddd-bdb3-79c69e4bd543 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:07.359856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf22da3-8f67-4718-bacd-dea061f04a4b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:07.371577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55254056-b7b2-46d3-b1b9-7bfb58ab10e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:07.372666 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55254056-b7b2-46d3-b1b9-7bfb58ab10e7 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:07.373161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55254056-b7b2-46d3-b1b9-7bfb58ab10e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:07.373645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55254056-b7b2-46d3-b1b9-7bfb58ab10e7 None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:07.385134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55254056-b7b2-46d3-b1b9-7bfb58ab10e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:07.387567 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55254056-b7b2-46d3-b1b9-7bfb58ab10e7 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:07.404549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:07.416850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7356aa-2f13-4327-945f-b216eeb4a47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:07.417666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7356aa-2f13-4327-945f-b216eeb4a47f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:07.417981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7356aa-2f13-4327-945f-b216eeb4a47f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:07.418249 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7356aa-2f13-4327-945f-b216eeb4a47f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:07.437135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 234/478] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:03:07 2022] POST /identity/v3/users => generated 249 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:07.473054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7356aa-2f13-4327-945f-b216eeb4a47f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-ae7356aa-2f13-4327-945f-b216eeb4a47f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:07.482983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:07.483955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:07.485655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:07.485655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:07.485655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:07.485655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:07.485655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:07.485655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:07.485655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:07.485655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:07.485655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:07.485655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:07.485655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:03:07.485655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:07.489923 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 245/479] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:03:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:07.496426 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55254056-b7b2-46d3-b1b9-7bfb58ab10e7 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:07.596572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:07.649723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55254056-b7b2-46d3-b1b9-7bfb58ab10e7 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:07.700184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b55d0-263b-4bc8-ac2d-7684b898087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:07.700818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b55d0-263b-4bc8-ac2d-7684b898087e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:07.701093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b55d0-263b-4bc8-ac2d-7684b898087e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:07.701360 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b55d0-263b-4bc8-ac2d-7684b898087e None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:07.702053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b55d0-263b-4bc8-ac2d-7684b898087e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:07.702352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b55d0-263b-4bc8-ac2d-7684b898087e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:07.704015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b55d0-263b-4bc8-ac2d-7684b898087e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:07.715028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 235/480] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:08.660665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7d31c-57aa-4945-b8cf-f50e55406b1a None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:08.661378 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7d31c-57aa-4945-b8cf-f50e55406b1a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:08.661378 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7d31c-57aa-4945-b8cf-f50e55406b1a None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:08.662873 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 246/481] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:08.668813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33f9fc-e449-4152-9496-afd39ba39f7c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:08.669474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33f9fc-e449-4152-9496-afd39ba39f7c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:08.669474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33f9fc-e449-4152-9496-afd39ba39f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:08.695256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-7c33f9fc-e449-4152-9496-afd39ba39f7c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:08.696655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7c33f9fc-e449-4152-9496-afd39ba39f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:08.791418 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 236/482] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:09.139214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7d31c-57aa-4945-b8cf-f50e55406b1a None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:09.270238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb162b3c-f225-48a7-856b-8ef0b2d9c863 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:09.270661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb162b3c-f225-48a7-856b-8ef0b2d9c863 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:09.270853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb162b3c-f225-48a7-856b-8ef0b2d9c863 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:09.272887 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 237/483] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:09.279319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42caba-53af-42a2-ab6e-ae887b2c0656 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:09.279633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42caba-53af-42a2-ab6e-ae887b2c0656 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:09.279887 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42caba-53af-42a2-ab6e-ae887b2c0656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:09.313661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7d31c-57aa-4945-b8cf-f50e55406b1a None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:09.321435 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-9c42caba-53af-42a2-ab6e-ae887b2c0656 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:09.323242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9c42caba-53af-42a2-ab6e-ae887b2c0656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:09.360386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353b697-34c8-421b-9a5e-f9bc63d36c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:09.361116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353b697-34c8-421b-9a5e-f9bc63d36c2d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:09.362137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353b697-34c8-421b-9a5e-f9bc63d36c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:09.362417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353b697-34c8-421b-9a5e-f9bc63d36c2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:09.376772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353b697-34c8-421b-9a5e-f9bc63d36c2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-4353b697-34c8-421b-9a5e-f9bc63d36c2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:09.382828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:09.385355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:09.387544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 247/484] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:09.396464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353b697-34c8-421b-9a5e-f9bc63d36c2d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:09.471090 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:09.522337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 238/485] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:03:09.560054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:09.736322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353b697-34c8-421b-9a5e-f9bc63d36c2d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:09.826330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401c046-5619-4d58-9a27-12e8cc10ae94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:09.827331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401c046-5619-4d58-9a27-12e8cc10ae94 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:09.827874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401c046-5619-4d58-9a27-12e8cc10ae94 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:09.828947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401c046-5619-4d58-9a27-12e8cc10ae94 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:09.829895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401c046-5619-4d58-9a27-12e8cc10ae94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:09.830463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401c046-5619-4d58-9a27-12e8cc10ae94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:09.832348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401c046-5619-4d58-9a27-12e8cc10ae94 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:09.854267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 248/486] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:09.860269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42caba-53af-42a2-ab6e-ae887b2c0656 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:09.960666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb3f8e-3eb5-4d41-abce-96209f8dd702 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:09.961334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb3f8e-3eb5-4d41-abce-96209f8dd702 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:09.961842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb3f8e-3eb5-4d41-abce-96209f8dd702 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:09.963541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 249/487] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:03:09.972027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25649f-9d50-4a1d-bd5e-a4e1eca54f24 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:09.972696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25649f-9d50-4a1d-bd5e-a4e1eca54f24 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:09.973212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25649f-9d50-4a1d-bd5e-a4e1eca54f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:10.004418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7d25649f-9d50-4a1d-bd5e-a4e1eca54f24 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:10.007482 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7d25649f-9d50-4a1d-bd5e-a4e1eca54f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:10.091148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42caba-53af-42a2-ab6e-ae887b2c0656 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:10.142716 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 250/488] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:03:10.150860 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25649f-9d50-4a1d-bd5e-a4e1eca54f24 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:10.217810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e25f5d-547c-489b-b503-5bdb67b04dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:10.218946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e25f5d-547c-489b-b503-5bdb67b04dee None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:10.219455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e25f5d-547c-489b-b503-5bdb67b04dee None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:10.219967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e25f5d-547c-489b-b503-5bdb67b04dee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:10.257359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e25f5d-547c-489b-b503-5bdb67b04dee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:10.259345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e25f5d-547c-489b-b503-5bdb67b04dee None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:10.262284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-c0e25f5d-547c-489b-b503-5bdb67b04dee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:10.308151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:10.309242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:10.310233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:10.310233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:10.310233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:10.313276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 239/489] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 453 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:10.321460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e25f5d-547c-489b-b503-5bdb67b04dee None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:10.401113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25649f-9d50-4a1d-bd5e-a4e1eca54f24 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:10.406528 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:10.459488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e956bf-035a-41b9-b74a-c461193e18eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:10.461129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e956bf-035a-41b9-b74a-c461193e18eb None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:10.461433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e956bf-035a-41b9-b74a-c461193e18eb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:10.461773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e956bf-035a-41b9-b74a-c461193e18eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:10.477579 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e956bf-035a-41b9-b74a-c461193e18eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-d4e956bf-035a-41b9-b74a-c461193e18eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:10.482981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:10.484010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:10.485506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:10.485506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:10.485506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:10.485506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:10.485506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:10.485506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:10.485506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:10.485506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:10.485506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:10.485506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:10.485506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:10.485506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:10.490316 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 251/490] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:10.491573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f48c2-3e26-4219-a344-8d5bde2aa2e1 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:10.491935 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f48c2-3e26-4219-a344-8d5bde2aa2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:10.492237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f48c2-3e26-4219-a344-8d5bde2aa2e1 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:10.493711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 252/491] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:10.496247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30685cff-b857-43a1-9008-580de08a17df None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:10.496247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30685cff-b857-43a1-9008-580de08a17df None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:10.496247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30685cff-b857-43a1-9008-580de08a17df None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:10.497328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 253/492] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:10.498733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30685cff-b857-43a1-9008-580de08a17df None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:10.510006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e25f5d-547c-489b-b503-5bdb67b04dee None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:10.553512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9ba55-ce6b-4fdb-bd78-986ad7d99109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:10.554616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9ba55-ce6b-4fdb-bd78-986ad7d99109 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:10.555081 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9ba55-ce6b-4fdb-bd78-986ad7d99109 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:10.555585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9ba55-ce6b-4fdb-bd78-986ad7d99109 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:10.557241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9ba55-ce6b-4fdb-bd78-986ad7d99109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:10.557241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9ba55-ce6b-4fdb-bd78-986ad7d99109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:10.559392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9ba55-ce6b-4fdb-bd78-986ad7d99109 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:10.577474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 240/493] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:10.579015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258420b-408b-4a6e-b0a8-c8956d3fd619 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:10.579322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258420b-408b-4a6e-b0a8-c8956d3fd619 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:10.579750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258420b-408b-4a6e-b0a8-c8956d3fd619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:10.605003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-5258420b-408b-4a6e-b0a8-c8956d3fd619 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:10.606357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5258420b-408b-4a6e-b0a8-c8956d3fd619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:10.630753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30685cff-b857-43a1-9008-580de08a17df None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:10.632539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:10.677732 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21d27f-0c35-414d-9a83-56a88f56b82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:10.678705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21d27f-0c35-414d-9a83-56a88f56b82d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:10.679140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21d27f-0c35-414d-9a83-56a88f56b82d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:10.679486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21d27f-0c35-414d-9a83-56a88f56b82d None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:10.680241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21d27f-0c35-414d-9a83-56a88f56b82d None admin] RBAC: Adding query filter params (name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:10.680539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21d27f-0c35-414d-9a83-56a88f56b82d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:10.682580 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21d27f-0c35-414d-9a83-56a88f56b82d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:10.685514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 241/494] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:10.688715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e624-1f74-4521-b287-cb8c6fe472da None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:10.689455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e624-1f74-4521-b287-cb8c6fe472da None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:10.690154 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e624-1f74-4521-b287-cb8c6fe472da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:10.696192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 254/495] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:10.697742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21d27f-0c35-414d-9a83-56a88f56b82d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:10.722057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-e6d4e624-1f74-4521-b287-cb8c6fe472da None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:10.724675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e6d4e624-1f74-4521-b287-cb8c6fe472da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:10.758451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:10.836036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 242/496] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:10.836036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4e624-1f74-4521-b287-cb8c6fe472da None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:10.837801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21d27f-0c35-414d-9a83-56a88f56b82d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:10.901826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81157d-fe72-4f2e-9cb1-93ed8d0fa110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:10.902704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81157d-fe72-4f2e-9cb1-93ed8d0fa110 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:10.903476 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81157d-fe72-4f2e-9cb1-93ed8d0fa110 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:10.903902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81157d-fe72-4f2e-9cb1-93ed8d0fa110 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:10.921252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81157d-fe72-4f2e-9cb1-93ed8d0fa110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-2e81157d-fe72-4f2e-9cb1-93ed8d0fa110 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:10.926965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:10.928115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:10.928977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 255/497] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:10.929709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81157d-fe72-4f2e-9cb1-93ed8d0fa110 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:10.963513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:11.019869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4e624-1f74-4521-b287-cb8c6fe472da None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.064503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6cbd3-38d8-4e40-9463-bf0b34df708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.065150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6cbd3-38d8-4e40-9463-bf0b34df708d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.065422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6cbd3-38d8-4e40-9463-bf0b34df708d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.065694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6cbd3-38d8-4e40-9463-bf0b34df708d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.073682 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6cbd3-38d8-4e40-9463-bf0b34df708d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.075189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6cbd3-38d8-4e40-9463-bf0b34df708d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.080461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81157d-fe72-4f2e-9cb1-93ed8d0fa110 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-84b6cbd3-38d8-4e40-9463-bf0b34df708d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:11.090612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:11.092083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:11.093095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:11.093095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:11.093095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:11.093095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:11.093095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:11.093095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:11.093095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:11.093095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:11.093095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:03:11.093095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:11.093095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 243/498] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Aug 30 01:03:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:11.093095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6cbd3-38d8-4e40-9463-bf0b34df708d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.120276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:11.128833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec8cdf-273a-458c-a8f9-c47b2988add5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.129567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec8cdf-273a-458c-a8f9-c47b2988add5 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.129930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec8cdf-273a-458c-a8f9-c47b2988add5 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.130320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec8cdf-273a-458c-a8f9-c47b2988add5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.144078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec8cdf-273a-458c-a8f9-c47b2988add5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.145803 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec8cdf-273a-458c-a8f9-c47b2988add5 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-d1ec8cdf-273a-458c-a8f9-c47b2988add5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:11.164649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:11.165811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:11.166645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:11.166645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:11.166645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:11.166645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 256/499] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:11.167055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec8cdf-273a-458c-a8f9-c47b2988add5 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.230899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6cbd3-38d8-4e40-9463-bf0b34df708d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.264176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:11.274895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8070cce-d01e-4bca-a4f2-945989881c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.275545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8070cce-d01e-4bca-a4f2-945989881c74 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.275838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8070cce-d01e-4bca-a4f2-945989881c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.276159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8070cce-d01e-4bca-a4f2-945989881c74 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.276900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8070cce-d01e-4bca-a4f2-945989881c74 None admin] RBAC: Adding query filter params (name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:11.277232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8070cce-d01e-4bca-a4f2-945989881c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.278787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8070cce-d01e-4bca-a4f2-945989881c74 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.288499 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 244/500] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:11.289404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8070cce-d01e-4bca-a4f2-945989881c74 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.293025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec8cdf-273a-458c-a8f9-c47b2988add5 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.332263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a203c4-a84e-41bd-99f4-e044db027e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.332908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a203c4-a84e-41bd-99f4-e044db027e7c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.333183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a203c4-a84e-41bd-99f4-e044db027e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.333458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a203c4-a84e-41bd-99f4-e044db027e7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.346259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a203c4-a84e-41bd-99f4-e044db027e7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-c7a203c4-a84e-41bd-99f4-e044db027e7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:11.351736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:11.352679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:11.353588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 257/501] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:11.354334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a203c4-a84e-41bd-99f4-e044db027e7c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.401506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:11.421455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8070cce-d01e-4bca-a4f2-945989881c74 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.467518 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fe31a-354e-4480-9654-490672c55564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.468511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fe31a-354e-4480-9654-490672c55564 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.469774 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fe31a-354e-4480-9654-490672c55564 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.470241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fe31a-354e-4480-9654-490672c55564 None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.471435 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fe31a-354e-4480-9654-490672c55564 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:11.471776 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fe31a-354e-4480-9654-490672c55564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.474042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fe31a-354e-4480-9654-490672c55564 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.502598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a203c4-a84e-41bd-99f4-e044db027e7c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.521269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 245/502] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Aug 30 01:03:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:11.522354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fe31a-354e-4480-9654-490672c55564 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.553280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760960e-5357-44bb-a12a-27d29fe2c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.554231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760960e-5357-44bb-a12a-27d29fe2c91e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.554703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760960e-5357-44bb-a12a-27d29fe2c91e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.555194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760960e-5357-44bb-a12a-27d29fe2c91e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.560272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:11.577078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760960e-5357-44bb-a12a-27d29fe2c91e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.579915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760960e-5357-44bb-a12a-27d29fe2c91e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-4760960e-5357-44bb-a12a-27d29fe2c91e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:11.611391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:11.612950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:11.613800 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:11.613800 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:11.613800 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:11.614415 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 258/503] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:11.616233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760960e-5357-44bb-a12a-27d29fe2c91e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.661970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fe31a-354e-4480-9654-490672c55564 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.704528 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f45d99-7028-4eaa-a52b-c5abf434a479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.705135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f45d99-7028-4eaa-a52b-c5abf434a479 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.705417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f45d99-7028-4eaa-a52b-c5abf434a479 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.705672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f45d99-7028-4eaa-a52b-c5abf434a479 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.706365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f45d99-7028-4eaa-a52b-c5abf434a479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:11.706704 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f45d99-7028-4eaa-a52b-c5abf434a479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.708263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f45d99-7028-4eaa-a52b-c5abf434a479 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.717960 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:11.724408 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 246/504] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:11.725717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f45d99-7028-4eaa-a52b-c5abf434a479 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.762473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760960e-5357-44bb-a12a-27d29fe2c91e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.800348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af031221-93c8-41a9-907a-3fb805598c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.801099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af031221-93c8-41a9-907a-3fb805598c0d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.801392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af031221-93c8-41a9-907a-3fb805598c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.801682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af031221-93c8-41a9-907a-3fb805598c0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.816146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af031221-93c8-41a9-907a-3fb805598c0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.817637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af031221-93c8-41a9-907a-3fb805598c0d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.832440 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f45d99-7028-4eaa-a52b-c5abf434a479 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-af031221-93c8-41a9-907a-3fb805598c0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:11.834077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:11.835060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:11.836010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:11.836010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:11.836010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:11.836010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 259/505] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:11.836235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af031221-93c8-41a9-907a-3fb805598c0d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.876417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb61b6-6dc2-42b1-8db7-908a191584c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:11.877133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb61b6-6dc2-42b1-8db7-908a191584c8 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:11.877460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb61b6-6dc2-42b1-8db7-908a191584c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:11.877736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb61b6-6dc2-42b1-8db7-908a191584c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:11.878421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb61b6-6dc2-42b1-8db7-908a191584c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:11.878770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb61b6-6dc2-42b1-8db7-908a191584c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:11.880273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb61b6-6dc2-42b1-8db7-908a191584c8 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:11.883820 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:11.889425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 247/506] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:11.890316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb61b6-6dc2-42b1-8db7-908a191584c8 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:11.957262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af031221-93c8-41a9-907a-3fb805598c0d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:11.973098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:12.003924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ae385-ceb0-415a-b477-d1f5e56a02c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.004611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ae385-ceb0-415a-b477-d1f5e56a02c1 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.005362 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ae385-ceb0-415a-b477-d1f5e56a02c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.005712 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ae385-ceb0-415a-b477-d1f5e56a02c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.042462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb61b6-6dc2-42b1-8db7-908a191584c8 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.085267 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69f972-308c-4f8f-af90-acd105611027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.085800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69f972-308c-4f8f-af90-acd105611027 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.086072 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69f972-308c-4f8f-af90-acd105611027 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.086342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69f972-308c-4f8f-af90-acd105611027 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.087067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69f972-308c-4f8f-af90-acd105611027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:12.087342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69f972-308c-4f8f-af90-acd105611027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.088764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69f972-308c-4f8f-af90-acd105611027 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.095840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ae385-ceb0-415a-b477-d1f5e56a02c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.098000 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ae385-ceb0-415a-b477-d1f5e56a02c1 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.106931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 248/507] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:12.106931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69f972-308c-4f8f-af90-acd105611027 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.174286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:12.213514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69f972-308c-4f8f-af90-acd105611027 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.239530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:12.254831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60877b0e-79d8-43cb-9743-255da9fb3fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.255763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60877b0e-79d8-43cb-9743-255da9fb3fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.256073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60877b0e-79d8-43cb-9743-255da9fb3fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.256366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60877b0e-79d8-43cb-9743-255da9fb3fc1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.265974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60877b0e-79d8-43cb-9743-255da9fb3fc1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.267972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60877b0e-79d8-43cb-9743-255da9fb3fc1 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.269045 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 260/508] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:03:12.270218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ae385-ceb0-415a-b477-d1f5e56a02c1 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-60877b0e-79d8-43cb-9743-255da9fb3fc1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:12.282675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:12.283783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:12.284883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:12.284883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:12.284883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:12.284883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:12.284883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:12.284883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:12.284883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:12.284883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:12.284883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 01:03:12.284883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:12.284883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 249/509] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:12.285674 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60877b0e-79d8-43cb-9743-255da9fb3fc1 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.417512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ae385-ceb0-415a-b477-d1f5e56a02c1 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.419665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60877b0e-79d8-43cb-9743-255da9fb3fc1 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.467968 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddce02a-592f-49da-a5ef-14e698add5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.468674 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddce02a-592f-49da-a5ef-14e698add5ed None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.468941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddce02a-592f-49da-a5ef-14e698add5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.469215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddce02a-592f-49da-a5ef-14e698add5ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.469364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad899fc-9488-4eeb-93b3-935f200c5bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.470309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad899fc-9488-4eeb-93b3-935f200c5bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.470815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad899fc-9488-4eeb-93b3-935f200c5bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.471318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad899fc-9488-4eeb-93b3-935f200c5bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.472540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad899fc-9488-4eeb-93b3-935f200c5bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:12.473084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad899fc-9488-4eeb-93b3-935f200c5bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.474948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad899fc-9488-4eeb-93b3-935f200c5bf7 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.481257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddce02a-592f-49da-a5ef-14e698add5ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.482867 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddce02a-592f-49da-a5ef-14e698add5ed None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.486539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:12.492172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 261/510] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:12.493304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad899fc-9488-4eeb-93b3-935f200c5bf7 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-0ddce02a-592f-49da-a5ef-14e698add5ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:12.498293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:12.504722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:12.506060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:12.506060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:12.506060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:12.506060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 250/511] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:12.506060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddce02a-592f-49da-a5ef-14e698add5ed None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.548331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:12.633992 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad899fc-9488-4eeb-93b3-935f200c5bf7 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.647478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddce02a-592f-49da-a5ef-14e698add5ed None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.681409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70c0e3-3bf2-4504-9460-bb9eb5bf4714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.684256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70c0e3-3bf2-4504-9460-bb9eb5bf4714 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.684256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70c0e3-3bf2-4504-9460-bb9eb5bf4714 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.684256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70c0e3-3bf2-4504-9460-bb9eb5bf4714 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.684702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70c0e3-3bf2-4504-9460-bb9eb5bf4714 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.685209 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644d763-c587-46b5-8a16-4f77ed00bd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.685897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644d763-c587-46b5-8a16-4f77ed00bd51 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.686188 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644d763-c587-46b5-8a16-4f77ed00bd51 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.686764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644d763-c587-46b5-8a16-4f77ed00bd51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.688694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70c0e3-3bf2-4504-9460-bb9eb5bf4714 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.699505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644d763-c587-46b5-8a16-4f77ed00bd51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.700994 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644d763-c587-46b5-8a16-4f77ed00bd51 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-f644d763-c587-46b5-8a16-4f77ed00bd51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:12.718421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:12.720177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:12.721036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:12.721036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:12.721036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:12.721036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 251/512] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:12.721036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644d763-c587-46b5-8a16-4f77ed00bd51 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.737368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 262/513] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:03:12 2022] POST /identity/v3/users => generated 247 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:12.737368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70c0e3-3bf2-4504-9460-bb9eb5bf4714 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.844923 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:12.855011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:12.881366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644d763-c587-46b5-8a16-4f77ed00bd51 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.904656 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70c0e3-3bf2-4504-9460-bb9eb5bf4714 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:12.927667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c5cb6-6dce-4eff-b538-f57f716429c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.928667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c5cb6-6dce-4eff-b538-f57f716429c3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.929061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c5cb6-6dce-4eff-b538-f57f716429c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.929320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c5cb6-6dce-4eff-b538-f57f716429c3 None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.930131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c5cb6-6dce-4eff-b538-f57f716429c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:12.930593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c5cb6-6dce-4eff-b538-f57f716429c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.932479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c5cb6-6dce-4eff-b538-f57f716429c3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.956512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 252/514] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Aug 30 01:03:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:12.957447 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c5cb6-6dce-4eff-b538-f57f716429c3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:12.969299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5a57b-ddef-4f04-8ff2-3dc57603b5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:12.969945 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5a57b-ddef-4f04-8ff2-3dc57603b5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:12.970221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5a57b-ddef-4f04-8ff2-3dc57603b5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:12.970522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5a57b-ddef-4f04-8ff2-3dc57603b5f8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:12.979916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5a57b-ddef-4f04-8ff2-3dc57603b5f8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:12.982192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5a57b-ddef-4f04-8ff2-3dc57603b5f8 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-aee5a57b-ddef-4f04-8ff2-3dc57603b5f8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:12.996949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:12.998528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:12.999916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:12.999916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:12.999916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:12.999916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:12.999916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:12.999916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:12.999916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:12.999916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:12.999916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:03:12.999916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:12.999916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 263/515] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:12.999916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5a57b-ddef-4f04-8ff2-3dc57603b5f8 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:13.108852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c5cb6-6dce-4eff-b538-f57f716429c3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:13.138248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5a57b-ddef-4f04-8ff2-3dc57603b5f8 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:13.154554 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997e000-fcdc-4ed4-82d6-94eeb0b99687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:13.155262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997e000-fcdc-4ed4-82d6-94eeb0b99687 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:13.155536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997e000-fcdc-4ed4-82d6-94eeb0b99687 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:13.155803 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997e000-fcdc-4ed4-82d6-94eeb0b99687 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:13.157333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997e000-fcdc-4ed4-82d6-94eeb0b99687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:13.157612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997e000-fcdc-4ed4-82d6-94eeb0b99687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:13.158969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997e000-fcdc-4ed4-82d6-94eeb0b99687 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:13.160395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:13.184801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 253/516] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:13.184801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997e000-fcdc-4ed4-82d6-94eeb0b99687 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:13.188571 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:13.194402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e1f2f-c51b-4c4a-b735-9e60972e609a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:13.195280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e1f2f-c51b-4c4a-b735-9e60972e609a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:13.195645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e1f2f-c51b-4c4a-b735-9e60972e609a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:13.195922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e1f2f-c51b-4c4a-b735-9e60972e609a None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:13.197110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e1f2f-c51b-4c4a-b735-9e60972e609a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:13.197689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e1f2f-c51b-4c4a-b735-9e60972e609a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:13.200201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e1f2f-c51b-4c4a-b735-9e60972e609a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:13.225506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 264/517] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:13.227023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e1f2f-c51b-4c4a-b735-9e60972e609a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:13.335089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997e000-fcdc-4ed4-82d6-94eeb0b99687 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:13.373237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f10b6-8184-4ef7-8e15-674f4b1d2bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:13.373860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f10b6-8184-4ef7-8e15-674f4b1d2bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:13.374131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f10b6-8184-4ef7-8e15-674f4b1d2bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:13.374399 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f10b6-8184-4ef7-8e15-674f4b1d2bc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:13.389678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e1f2f-c51b-4c4a-b735-9e60972e609a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:13.453518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a46d8-5732-466a-baf3-06df1c23c782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:13.454214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a46d8-5732-466a-baf3-06df1c23c782 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:13.454587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a46d8-5732-466a-baf3-06df1c23c782 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:13.454855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a46d8-5732-466a-baf3-06df1c23c782 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:13.455629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a46d8-5732-466a-baf3-06df1c23c782 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:13.456104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a46d8-5732-466a-baf3-06df1c23c782 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:13.457537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a46d8-5732-466a-baf3-06df1c23c782 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:13.489038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f10b6-8184-4ef7-8e15-674f4b1d2bc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:13.492161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f10b6-8184-4ef7-8e15-674f4b1d2bc3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:13.500742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:13.501472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 265/518] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Aug 30 01:03:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:13.502878 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a46d8-5732-466a-baf3-06df1c23c782 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:13.567794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:13.652277 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 254/519] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:03:13.653143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f10b6-8184-4ef7-8e15-674f4b1d2bc3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:13.675769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a46d8-5732-466a-baf3-06df1c23c782 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:13.736922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfd20a-04d1-46b5-802b-f53b2fb6d5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:13.737584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfd20a-04d1-46b5-802b-f53b2fb6d5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:13.737843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfd20a-04d1-46b5-802b-f53b2fb6d5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:13.738093 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfd20a-04d1-46b5-802b-f53b2fb6d5f5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:13.746570 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfd20a-04d1-46b5-802b-f53b2fb6d5f5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:13.748202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfd20a-04d1-46b5-802b-f53b2fb6d5f5 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-2cdfd20a-04d1-46b5-802b-f53b2fb6d5f5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:13.761331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:13.762297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:13.763183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:13.763183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:13.763183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:13.763183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:13.763183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:13.763183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:13.763183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:13.763183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:13.763183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 01:03:13.763183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:13.763183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 266/520] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:13.763731 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfd20a-04d1-46b5-802b-f53b2fb6d5f5 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:13.816451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f10b6-8184-4ef7-8e15-674f4b1d2bc3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:13.821198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:13.859040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14fdb5-6671-4c4f-96ea-17bb5ebcff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:13.860074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14fdb5-6671-4c4f-96ea-17bb5ebcff1b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:13.860307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14fdb5-6671-4c4f-96ea-17bb5ebcff1b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:13.860573 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14fdb5-6671-4c4f-96ea-17bb5ebcff1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:13.869060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14fdb5-6671-4c4f-96ea-17bb5ebcff1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:13.870773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14fdb5-6671-4c4f-96ea-17bb5ebcff1b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-ab14fdb5-6671-4c4f-96ea-17bb5ebcff1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:13.885796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:13.886837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:13.889066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:13.889066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:13.889066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:13.889066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:13.889066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:13.889066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:13.889066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:13.889066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:13.889066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 255/521] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:13.889066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14fdb5-6671-4c4f-96ea-17bb5ebcff1b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:13.909256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:13.917728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfd20a-04d1-46b5-802b-f53b2fb6d5f5 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:13.971047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29143ae-17b1-4289-9a93-ef6634737d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:13.972250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29143ae-17b1-4289-9a93-ef6634737d13 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:13.973457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29143ae-17b1-4289-9a93-ef6634737d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:13.973873 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29143ae-17b1-4289-9a93-ef6634737d13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:14.027354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14fdb5-6671-4c4f-96ea-17bb5ebcff1b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:14.068229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fae4cc-0f19-49f4-895c-5fd67639de69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:14.069090 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29143ae-17b1-4289-9a93-ef6634737d13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:14.069460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fae4cc-0f19-49f4-895c-5fd67639de69 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:14.069952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fae4cc-0f19-49f4-895c-5fd67639de69 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:14.070414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fae4cc-0f19-49f4-895c-5fd67639de69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:14.071141 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29143ae-17b1-4289-9a93-ef6634737d13 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:14.083259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fae4cc-0f19-49f4-895c-5fd67639de69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:14.085093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fae4cc-0f19-49f4-895c-5fd67639de69 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-33fae4cc-0f19-49f4-895c-5fd67639de69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:14.101282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:14.102417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:14.103258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:14.103258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:14.103258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:14.109029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 256/522] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:14.109029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fae4cc-0f19-49f4-895c-5fd67639de69 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:14.128235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:14.192682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:14.206277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 267/523] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:03:14.206277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29143ae-17b1-4289-9a93-ef6634737d13 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:14.246323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fae4cc-0f19-49f4-895c-5fd67639de69 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:14.288775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040f7ba-e38e-4ec8-8888-6511f4b39fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:14.289799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040f7ba-e38e-4ec8-8888-6511f4b39fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:14.290216 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040f7ba-e38e-4ec8-8888-6511f4b39fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:14.290836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040f7ba-e38e-4ec8-8888-6511f4b39fe7 None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:14.291807 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040f7ba-e38e-4ec8-8888-6511f4b39fe7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:14.292336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040f7ba-e38e-4ec8-8888-6511f4b39fe7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:14.294291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040f7ba-e38e-4ec8-8888-6511f4b39fe7 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:14.320778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 257/524] 10.222.0.44 () {56 vars in 1139 bytes} [Tue Aug 30 01:03:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:14.322637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040f7ba-e38e-4ec8-8888-6511f4b39fe7 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:14.349780 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29143ae-17b1-4289-9a93-ef6634737d13 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:14.392380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a426faf-116d-418b-a1e9-7f5d32bbcc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:14.392915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a426faf-116d-418b-a1e9-7f5d32bbcc9a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:14.393194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a426faf-116d-418b-a1e9-7f5d32bbcc9a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:14.393473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a426faf-116d-418b-a1e9-7f5d32bbcc9a None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:14.394222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a426faf-116d-418b-a1e9-7f5d32bbcc9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:14.394531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a426faf-116d-418b-a1e9-7f5d32bbcc9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:14.396136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a426faf-116d-418b-a1e9-7f5d32bbcc9a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:14.415717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 268/525] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:14.416527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a426faf-116d-418b-a1e9-7f5d32bbcc9a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:14.470614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:14.493672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:14.518110 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040f7ba-e38e-4ec8-8888-6511f4b39fe7 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:14.566335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a426faf-116d-418b-a1e9-7f5d32bbcc9a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:14.575337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24303fb-45d4-4988-83ed-f8da49ca09f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:14.575927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24303fb-45d4-4988-83ed-f8da49ca09f5 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:14.576280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24303fb-45d4-4988-83ed-f8da49ca09f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:14.576845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24303fb-45d4-4988-83ed-f8da49ca09f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:14.577929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24303fb-45d4-4988-83ed-f8da49ca09f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:14.579233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24303fb-45d4-4988-83ed-f8da49ca09f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:14.580143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24303fb-45d4-4988-83ed-f8da49ca09f5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:14.594379 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 258/526] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:14.595181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24303fb-45d4-4988-83ed-f8da49ca09f5 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:14.626510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e048e-8660-4350-8c46-27e4090fff6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:14.627302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e048e-8660-4350-8c46-27e4090fff6d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:14.627663 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e048e-8660-4350-8c46-27e4090fff6d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:14.627940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e048e-8660-4350-8c46-27e4090fff6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:14.648206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e048e-8660-4350-8c46-27e4090fff6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:14.650594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e048e-8660-4350-8c46-27e4090fff6d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-c61e048e-8660-4350-8c46-27e4090fff6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:14.674650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:14.676137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:14.678001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:14.678001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:14.678001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:14.678001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 269/527] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:14.678001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e048e-8660-4350-8c46-27e4090fff6d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:14.759004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24303fb-45d4-4988-83ed-f8da49ca09f5 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:14.808483 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fcade-6cbb-4d4d-8520-14dfb4eb0b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:14.809575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fcade-6cbb-4d4d-8520-14dfb4eb0b7d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:14.809959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fcade-6cbb-4d4d-8520-14dfb4eb0b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:14.810378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fcade-6cbb-4d4d-8520-14dfb4eb0b7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:14.814125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:14.826613 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e048e-8660-4350-8c46-27e4090fff6d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:14.837423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:14.873128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07cb9b-76a7-48e0-916f-1605c05b1079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:14.874029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07cb9b-76a7-48e0-916f-1605c05b1079 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:14.874472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07cb9b-76a7-48e0-916f-1605c05b1079 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:14.874742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07cb9b-76a7-48e0-916f-1605c05b1079 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:14.875562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07cb9b-76a7-48e0-916f-1605c05b1079 None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=049399a752ed4646a13f8e4a3a62d8e9) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:14.876520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07cb9b-76a7-48e0-916f-1605c05b1079 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:14.878118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07cb9b-76a7-48e0-916f-1605c05b1079 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:14.884793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 270/528] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:14 2022] GET /identity/v3/role_assignments?user.id=049399a752ed4646a13f8e4a3a62d8e9&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:14.885503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07cb9b-76a7-48e0-916f-1605c05b1079 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:14.919606 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7fcade-6cbb-4d4d-8520-14dfb4eb0b7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:14.924378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7fcade-6cbb-4d4d-8520-14dfb4eb0b7d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:15.041552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07cb9b-76a7-48e0-916f-1605c05b1079 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:15.074873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 259/529] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:03:15.076304 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fcade-6cbb-4d4d-8520-14dfb4eb0b7d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:15.083281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf11dd1-b0fd-49dd-9a72-8760c4dd54c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:15.083936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf11dd1-b0fd-49dd-9a72-8760c4dd54c4 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:15.084271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf11dd1-b0fd-49dd-9a72-8760c4dd54c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:15.084540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf11dd1-b0fd-49dd-9a72-8760c4dd54c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:15.092358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf11dd1-b0fd-49dd-9a72-8760c4dd54c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:15.094606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf11dd1-b0fd-49dd-9a72-8760c4dd54c4 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-6bf11dd1-b0fd-49dd-9a72-8760c4dd54c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:15.108391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:15.109692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:15.110705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:15.110705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:15.110705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:15.110705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:15.110705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:15.110705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:15.110705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:15.110705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:15.110705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 271/530] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:15.112337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33e16a-2fc0-4285-878e-6627817bbc20 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:15.112337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33e16a-2fc0-4285-878e-6627817bbc20 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:15.112337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33e16a-2fc0-4285-878e-6627817bbc20 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:15.113186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 272/531] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:15.118257 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33e16a-2fc0-4285-878e-6627817bbc20 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:15.125951 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:15.158456 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:15.214233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fcade-6cbb-4d4d-8520-14dfb4eb0b7d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:15.261889 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33e16a-2fc0-4285-878e-6627817bbc20 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:15.262890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a1f46-e3bc-42b3-8fff-95cee8795f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:15.263719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a1f46-e3bc-42b3-8fff-95cee8795f36 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:15.264206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a1f46-e3bc-42b3-8fff-95cee8795f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:15.264919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a1f46-e3bc-42b3-8fff-95cee8795f36 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:15.266289 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a1f46-e3bc-42b3-8fff-95cee8795f36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:15.267775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a1f46-e3bc-42b3-8fff-95cee8795f36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:15.269452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a1f46-e3bc-42b3-8fff-95cee8795f36 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:15.294137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 260/532] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:15.295242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a1f46-e3bc-42b3-8fff-95cee8795f36 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:15.314694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ae8c0-6aa0-4590-8a21-dd26d20c009b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:15.315329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ae8c0-6aa0-4590-8a21-dd26d20c009b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:15.315700 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ae8c0-6aa0-4590-8a21-dd26d20c009b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:15.316031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ae8c0-6aa0-4590-8a21-dd26d20c009b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:15.334597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ae8c0-6aa0-4590-8a21-dd26d20c009b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:15.336962 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ae8c0-6aa0-4590-8a21-dd26d20c009b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-d02ae8c0-6aa0-4590-8a21-dd26d20c009b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:15.361123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:15.362065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:15.362902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:15.362902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:15.362902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:15.362902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 273/533] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:15.364570 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59ed77-a074-4eaf-aace-14eb64a42ac8 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:15.365051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59ed77-a074-4eaf-aace-14eb64a42ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:15.365522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59ed77-a074-4eaf-aace-14eb64a42ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:15.388346 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-2f59ed77-a074-4eaf-aace-14eb64a42ac8 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:15.390109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2f59ed77-a074-4eaf-aace-14eb64a42ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:15.446983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:15.456362 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a1f46-e3bc-42b3-8fff-95cee8795f36 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:15.474280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 274/534] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:15.475741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59ed77-a074-4eaf-aace-14eb64a42ac8 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:15.500483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:15.501468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91ab2d-58af-434c-86c6-04d9a0c50c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:15.502600 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91ab2d-58af-434c-86c6-04d9a0c50c19 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:15.503070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91ab2d-58af-434c-86c6-04d9a0c50c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:15.503537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91ab2d-58af-434c-86c6-04d9a0c50c19 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:15.504486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91ab2d-58af-434c-86c6-04d9a0c50c19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:15.505056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91ab2d-58af-434c-86c6-04d9a0c50c19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:15.506801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91ab2d-58af-434c-86c6-04d9a0c50c19 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:15.533541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 261/535] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:15.535084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91ab2d-58af-434c-86c6-04d9a0c50c19 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:15.641124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59ed77-a074-4eaf-aace-14eb64a42ac8 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:15.651150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91ab2d-58af-434c-86c6-04d9a0c50c19 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:15.695618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e5491-9a11-448d-b3b2-6dd0a3772d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:15.696253 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e5491-9a11-448d-b3b2-6dd0a3772d86 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:15.696523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e5491-9a11-448d-b3b2-6dd0a3772d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:15.696779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e5491-9a11-448d-b3b2-6dd0a3772d86 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:15.697421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e5491-9a11-448d-b3b2-6dd0a3772d86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:15.697716 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e5491-9a11-448d-b3b2-6dd0a3772d86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:15.699074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e5491-9a11-448d-b3b2-6dd0a3772d86 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:15.703007 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875180c-bb97-4a18-8eb7-5512e85afc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:15.704113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875180c-bb97-4a18-8eb7-5512e85afc01 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:15.704644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875180c-bb97-4a18-8eb7-5512e85afc01 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:15.705148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875180c-bb97-4a18-8eb7-5512e85afc01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:15.715857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 262/536] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:15.716965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e5491-9a11-448d-b3b2-6dd0a3772d86 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:15.718190 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c875180c-bb97-4a18-8eb7-5512e85afc01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:15.719685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c875180c-bb97-4a18-8eb7-5512e85afc01 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:15.739006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-c875180c-bb97-4a18-8eb7-5512e85afc01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:15.741065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:15.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:15.743461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:15.743461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:15.743461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:15.743461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:15.743461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:15.743461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:15.743461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:15.743461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:15.751368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 275/537] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:15.753349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875180c-bb97-4a18-8eb7-5512e85afc01 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:15.833635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:15.846758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e5491-9a11-448d-b3b2-6dd0a3772d86 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:15.867181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875180c-bb97-4a18-8eb7-5512e85afc01 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:15.884937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f85fd-d123-419e-9d95-f1ffe14606b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:15.885539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f85fd-d123-419e-9d95-f1ffe14606b6 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:15.885812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f85fd-d123-419e-9d95-f1ffe14606b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:15.886079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f85fd-d123-419e-9d95-f1ffe14606b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:15.886810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f85fd-d123-419e-9d95-f1ffe14606b6 None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=92c9d444b3294ed18c3d488cf0a3eeb2) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:15.887118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f85fd-d123-419e-9d95-f1ffe14606b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:15.888653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f85fd-d123-419e-9d95-f1ffe14606b6 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:15.894881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 263/538] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:15 2022] GET /identity/v3/role_assignments?user.id=92c9d444b3294ed18c3d488cf0a3eeb2&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:15.895715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f85fd-d123-419e-9d95-f1ffe14606b6 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:15.918989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d6d7d-40fd-4352-87dc-6a3afe347aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:15.919735 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d6d7d-40fd-4352-87dc-6a3afe347aac None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:15.920310 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d6d7d-40fd-4352-87dc-6a3afe347aac None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:15.920577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d6d7d-40fd-4352-87dc-6a3afe347aac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:15.929722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d6d7d-40fd-4352-87dc-6a3afe347aac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:15.931235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d6d7d-40fd-4352-87dc-6a3afe347aac None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-551d6d7d-40fd-4352-87dc-6a3afe347aac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:15.945439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:15.946385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:15.947240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:15.947240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:15.947240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:15.947240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:15.947240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:15.947240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:15.947240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:15.947240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:15.951469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 276/539] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:15.952597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d6d7d-40fd-4352-87dc-6a3afe347aac None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:16.025151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:16.059537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f85fd-d123-419e-9d95-f1ffe14606b6 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:16.085478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d6d7d-40fd-4352-87dc-6a3afe347aac None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:16.100306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b2dbb-c83b-4d7b-bb04-d7f6907e9a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:16.101125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b2dbb-c83b-4d7b-bb04-d7f6907e9a68 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:16.101476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b2dbb-c83b-4d7b-bb04-d7f6907e9a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:16.101824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b2dbb-c83b-4d7b-bb04-d7f6907e9a68 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:16.102593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b2dbb-c83b-4d7b-bb04-d7f6907e9a68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:16.102971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b2dbb-c83b-4d7b-bb04-d7f6907e9a68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:16.104612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b2dbb-c83b-4d7b-bb04-d7f6907e9a68 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:16.123179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 264/540] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:16.123179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b2dbb-c83b-4d7b-bb04-d7f6907e9a68 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:16.134930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:16.143120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788b480-160c-4c0f-8806-97dd5c6b8651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:16.144086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788b480-160c-4c0f-8806-97dd5c6b8651 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:16.144576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788b480-160c-4c0f-8806-97dd5c6b8651 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:16.145098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788b480-160c-4c0f-8806-97dd5c6b8651 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:16.160839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0788b480-160c-4c0f-8806-97dd5c6b8651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-0788b480-160c-4c0f-8806-97dd5c6b8651 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:16.168065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:16.168958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:16.170033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:16.170033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:16.170033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:16.170033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:16.170033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:16.170033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:16.170033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:16.170033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:16.170033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:16.170033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:16.170033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:16.170033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:16.173552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 277/541] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:16.174749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788b480-160c-4c0f-8806-97dd5c6b8651 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:16.246973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b2dbb-c83b-4d7b-bb04-d7f6907e9a68 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:16.286650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756eb1e-8ff1-475f-bd15-d2717235cd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:16.287364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756eb1e-8ff1-475f-bd15-d2717235cd2b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:16.287649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756eb1e-8ff1-475f-bd15-d2717235cd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:16.288033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756eb1e-8ff1-475f-bd15-d2717235cd2b None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:16.288739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756eb1e-8ff1-475f-bd15-d2717235cd2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:16.289044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756eb1e-8ff1-475f-bd15-d2717235cd2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:16.290619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756eb1e-8ff1-475f-bd15-d2717235cd2b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:16.297987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788b480-160c-4c0f-8806-97dd5c6b8651 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:16.299587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:16.306647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 265/542] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:16.307912 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756eb1e-8ff1-475f-bd15-d2717235cd2b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:16.336468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2145fb-8593-409d-88ff-61182e4293c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:16.337091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2145fb-8593-409d-88ff-61182e4293c0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:16.337371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2145fb-8593-409d-88ff-61182e4293c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:16.337649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2145fb-8593-409d-88ff-61182e4293c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:16.338772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2145fb-8593-409d-88ff-61182e4293c0 None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=534da5b47dda43dabaab2cb6aa683342) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:16.339262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2145fb-8593-409d-88ff-61182e4293c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:16.341091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2145fb-8593-409d-88ff-61182e4293c0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:16.349768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 278/543] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:16 2022] GET /identity/v3/role_assignments?user.id=534da5b47dda43dabaab2cb6aa683342&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:16.351464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2145fb-8593-409d-88ff-61182e4293c0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:16.419305 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:16.452302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756eb1e-8ff1-475f-bd15-d2717235cd2b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:16.479501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2145fb-8593-409d-88ff-61182e4293c0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:16.522201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070f59d-1750-47ac-9204-b08986ab40fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:16.522763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a89255-4c52-49b8-8b6e-2ec24aae907d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:16.522903 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070f59d-1750-47ac-9204-b08986ab40fd None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:16.523107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070f59d-1750-47ac-9204-b08986ab40fd None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:16.523386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070f59d-1750-47ac-9204-b08986ab40fd None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:16.524117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070f59d-1750-47ac-9204-b08986ab40fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:16.524408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a89255-4c52-49b8-8b6e-2ec24aae907d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:16.524521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070f59d-1750-47ac-9204-b08986ab40fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:16.524977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a89255-4c52-49b8-8b6e-2ec24aae907d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:16.525481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a89255-4c52-49b8-8b6e-2ec24aae907d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:16.525850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070f59d-1750-47ac-9204-b08986ab40fd None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:16.526694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a89255-4c52-49b8-8b6e-2ec24aae907d None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=86cea2f9238b42e0a7cb26bac494a8c7) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:16.527346 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a89255-4c52-49b8-8b6e-2ec24aae907d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:16.529298 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a89255-4c52-49b8-8b6e-2ec24aae907d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:16.538163 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 279/544] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:16 2022] GET /identity/v3/role_assignments?user.id=86cea2f9238b42e0a7cb26bac494a8c7&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:16.548298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 266/545] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:16.557581 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a89255-4c52-49b8-8b6e-2ec24aae907d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:16.681617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a89255-4c52-49b8-8b6e-2ec24aae907d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:16.705723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:16.731452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4702a9-8d0e-4a12-a782-b218da4b05f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:16.732289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4702a9-8d0e-4a12-a782-b218da4b05f5 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:16.732670 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4702a9-8d0e-4a12-a782-b218da4b05f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:16.733040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4702a9-8d0e-4a12-a782-b218da4b05f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:16.746440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4702a9-8d0e-4a12-a782-b218da4b05f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:16.748094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4702a9-8d0e-4a12-a782-b218da4b05f5 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-9c4702a9-8d0e-4a12-a782-b218da4b05f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:16.764328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:16.765603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:16.766865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:16.766865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:16.766865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:16.768263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 280/546] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:16.773705 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070f59d-1750-47ac-9204-b08986ab40fd None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:16.848244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:16.905432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070f59d-1750-47ac-9204-b08986ab40fd None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:16.950604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4be5b8-6049-47b2-83a2-7eabe608f0ba None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:16.950915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4be5b8-6049-47b2-83a2-7eabe608f0ba None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:16.951092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4be5b8-6049-47b2-83a2-7eabe608f0ba None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:16.953307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 281/547] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:03:16.960224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2607463-ee32-4bf0-b79a-f5708596fa12 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:16.960530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2607463-ee32-4bf0-b79a-f5708596fa12 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:16.960758 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2607463-ee32-4bf0-b79a-f5708596fa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:16.971188 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94bdc7-78d3-47a9-8514-94276244bf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:16.971808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94bdc7-78d3-47a9-8514-94276244bf35 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:16.972105 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94bdc7-78d3-47a9-8514-94276244bf35 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:16.972372 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94bdc7-78d3-47a9-8514-94276244bf35 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:16.973017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94bdc7-78d3-47a9-8514-94276244bf35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:16.973315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94bdc7-78d3-47a9-8514-94276244bf35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:16.974683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94bdc7-78d3-47a9-8514-94276244bf35 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:16.996145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b2607463-ee32-4bf0-b79a-f5708596fa12 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:16.997344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b2607463-ee32-4bf0-b79a-f5708596fa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:17.004173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 267/548] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:17.012827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94bdc7-78d3-47a9-8514-94276244bf35 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:17.099704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 282/549] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:17.185694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94bdc7-78d3-47a9-8514-94276244bf35 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:17.217817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:17.240910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bb3e8-c06e-467a-933e-0c00c5182e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:17.241777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bb3e8-c06e-467a-933e-0c00c5182e1d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:17.242222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bb3e8-c06e-467a-933e-0c00c5182e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:17.243059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bb3e8-c06e-467a-933e-0c00c5182e1d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:17.255948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bb3e8-c06e-467a-933e-0c00c5182e1d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:17.258068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bb3e8-c06e-467a-933e-0c00c5182e1d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-eb2bb3e8-c06e-467a-933e-0c00c5182e1d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:17.274723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:17.276079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:17.277134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:17.277134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:17.277134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:17.277134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:17.277134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:17.277134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:17.277134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:17.277134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:17.277134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:03:17.277134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:17.284012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 268/550] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:17.285328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2607463-ee32-4bf0-b79a-f5708596fa12 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:17.419219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2607463-ee32-4bf0-b79a-f5708596fa12 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:17.421105 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:17.457301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba0ef6-9201-4c87-84bf-8a1470e991da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:17.457925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba0ef6-9201-4c87-84bf-8a1470e991da None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:17.458205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba0ef6-9201-4c87-84bf-8a1470e991da None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:17.458477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba0ef6-9201-4c87-84bf-8a1470e991da None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:17.459213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba0ef6-9201-4c87-84bf-8a1470e991da None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:17.459519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba0ef6-9201-4c87-84bf-8a1470e991da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:17.460999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba0ef6-9201-4c87-84bf-8a1470e991da None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:17.480826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 283/551] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Aug 30 01:03:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:17.486812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bb3e8-c06e-467a-933e-0c00c5182e1d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:17.555893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba0ef6-9201-4c87-84bf-8a1470e991da None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:17.661880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bb3e8-c06e-467a-933e-0c00c5182e1d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:17.714847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390699b7-d17e-4634-a4ef-2884d4bac542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:17.715575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390699b7-d17e-4634-a4ef-2884d4bac542 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:17.715862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390699b7-d17e-4634-a4ef-2884d4bac542 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:17.716266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390699b7-d17e-4634-a4ef-2884d4bac542 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:17.760927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba0ef6-9201-4c87-84bf-8a1470e991da None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:17.814599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3dfd1-5683-461b-a86e-61cfccd0a70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:17.820419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3dfd1-5683-461b-a86e-61cfccd0a70a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:17.820701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3dfd1-5683-461b-a86e-61cfccd0a70a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:17.820976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3dfd1-5683-461b-a86e-61cfccd0a70a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:17.848724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390699b7-d17e-4634-a4ef-2884d4bac542 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:17.851676 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390699b7-d17e-4634-a4ef-2884d4bac542 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:17.864702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3dfd1-5683-461b-a86e-61cfccd0a70a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-16a3dfd1-5683-461b-a86e-61cfccd0a70a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:17.869628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:17.870664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:17.871895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 284/552] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:17.881050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3dfd1-5683-461b-a86e-61cfccd0a70a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:17.892393 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:17.942622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:18.042149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3dfd1-5683-461b-a86e-61cfccd0a70a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:18.086213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 269/553] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:03:18.092218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390699b7-d17e-4634-a4ef-2884d4bac542 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:18.126435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60dc7dc-d85a-41cf-9286-c4449672840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:18.127087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60dc7dc-d85a-41cf-9286-c4449672840f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.127321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60dc7dc-d85a-41cf-9286-c4449672840f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.127591 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60dc7dc-d85a-41cf-9286-c4449672840f None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.128308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60dc7dc-d85a-41cf-9286-c4449672840f None admin] RBAC: Adding query filter params (name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:18.128612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60dc7dc-d85a-41cf-9286-c4449672840f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:18.130009 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60dc7dc-d85a-41cf-9286-c4449672840f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:18.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 285/554] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:18.153409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60dc7dc-d85a-41cf-9286-c4449672840f None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:18.264365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:18.300720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390699b7-d17e-4634-a4ef-2884d4bac542 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:18.320372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:18.351839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60dc7dc-d85a-41cf-9286-c4449672840f None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:18.368707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e80bd6-c8f9-427c-a3ec-953677bbcc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:18.369356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e80bd6-c8f9-427c-a3ec-953677bbcc9c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.369632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e80bd6-c8f9-427c-a3ec-953677bbcc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.369901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e80bd6-c8f9-427c-a3ec-953677bbcc9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.390256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e80bd6-c8f9-427c-a3ec-953677bbcc9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:18.392187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e80bd6-c8f9-427c-a3ec-953677bbcc9c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-a9e80bd6-c8f9-427c-a3ec-953677bbcc9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:18.418509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:18.419489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:18.420646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:18.420646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:18.420646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:18.424384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1e99f-8331-4eff-97fa-ee0a2f249f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:18.425032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1e99f-8331-4eff-97fa-ee0a2f249f93 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.425379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1e99f-8331-4eff-97fa-ee0a2f249f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.425652 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1e99f-8331-4eff-97fa-ee0a2f249f93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.427608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 270/555] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:18.428539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e80bd6-c8f9-427c-a3ec-953677bbcc9c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:18.446404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1e99f-8331-4eff-97fa-ee0a2f249f93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:18.447887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1e99f-8331-4eff-97fa-ee0a2f249f93 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-cab1e99f-8331-4eff-97fa-ee0a2f249f93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:18.466559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:18.469385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:18.471581 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:18.471581 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:18.471581 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:18.471581 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 286/556] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:18.475326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e66c4a-1ffa-4a94-b0d6-0ad94968a7b3 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.475326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e66c4a-1ffa-4a94-b0d6-0ad94968a7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.475326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e66c4a-1ffa-4a94-b0d6-0ad94968a7b3 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.475326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 287/557] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:18.477987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e66c4a-1ffa-4a94-b0d6-0ad94968a7b3 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:18.600706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e66c4a-1ffa-4a94-b0d6-0ad94968a7b3 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:18.627054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e80bd6-c8f9-427c-a3ec-953677bbcc9c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:18.638587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcafda-9f0e-4c57-82e4-324d65214ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:18.639202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcafda-9f0e-4c57-82e4-324d65214ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.639480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcafda-9f0e-4c57-82e4-324d65214ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.639752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcafda-9f0e-4c57-82e4-324d65214ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.640614 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcafda-9f0e-4c57-82e4-324d65214ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:18.641144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcafda-9f0e-4c57-82e4-324d65214ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:18.642742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcafda-9f0e-4c57-82e4-324d65214ed4 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:18.644305 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:18.663802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 288/558] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:18.665453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69fbdf-c404-4ed1-8819-db268197e22b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.665897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69fbdf-c404-4ed1-8819-db268197e22b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.666329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69fbdf-c404-4ed1-8819-db268197e22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.684527 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:18.688155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-3e69fbdf-c404-4ed1-8819-db268197e22b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:18.689873 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3e69fbdf-c404-4ed1-8819-db268197e22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:18.692891 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56157c-b693-41f1-9288-b4f7c451cef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:18.694019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56157c-b693-41f1-9288-b4f7c451cef6 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.695320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56157c-b693-41f1-9288-b4f7c451cef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.695320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56157c-b693-41f1-9288-b4f7c451cef6 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.696104 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56157c-b693-41f1-9288-b4f7c451cef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:18.696650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56157c-b693-41f1-9288-b4f7c451cef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:18.699357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56157c-b693-41f1-9288-b4f7c451cef6 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:18.719206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 271/559] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:18.720158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56157c-b693-41f1-9288-b4f7c451cef6 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:18.773708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 289/560] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:18.774980 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69fbdf-c404-4ed1-8819-db268197e22b None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:18.855398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56157c-b693-41f1-9288-b4f7c451cef6 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:18.890805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69fbdf-c404-4ed1-8819-db268197e22b None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:18.895093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a8e1f-cadb-4312-9ece-de0b7fc7a702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:18.895738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a8e1f-cadb-4312-9ece-de0b7fc7a702 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.896096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a8e1f-cadb-4312-9ece-de0b7fc7a702 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.896377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a8e1f-cadb-4312-9ece-de0b7fc7a702 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.904738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a8e1f-cadb-4312-9ece-de0b7fc7a702 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:18.907018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a8e1f-cadb-4312-9ece-de0b7fc7a702 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:18.920823 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-974a8e1f-cadb-4312-9ece-de0b7fc7a702 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:18.921025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:18.922696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:18.923608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:18.923608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:18.923608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:18.923608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:18.923608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:18.923608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:18.923608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:18.923608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:18.923608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:03:18.923608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:18.923608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 272/561] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Aug 30 01:03:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:18.923608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229db3e-abc5-4063-bbee-529b75dcdb4b None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.924388 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229db3e-abc5-4063-bbee-529b75dcdb4b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.924388 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229db3e-abc5-4063-bbee-529b75dcdb4b None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.929406 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 273/562] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:18.930652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc252f-6957-4cbc-97b9-13847cd0316c None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.930931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc252f-6957-4cbc-97b9-13847cd0316c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.931204 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc252f-6957-4cbc-97b9-13847cd0316c None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.932529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 274/563] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:18.933233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc252f-6957-4cbc-97b9-13847cd0316c None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:18.938268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f143fe-92ce-4c76-b2a0-344da557aad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:18.939138 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f143fe-92ce-4c76-b2a0-344da557aad7 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.939560 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f143fe-92ce-4c76-b2a0-344da557aad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.939954 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f143fe-92ce-4c76-b2a0-344da557aad7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.947729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f143fe-92ce-4c76-b2a0-344da557aad7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:18.949535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f143fe-92ce-4c76-b2a0-344da557aad7 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-e8f143fe-92ce-4c76-b2a0-344da557aad7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:18.963011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:18.964149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:18.965228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:18.965228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:18.965228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:18.965228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:18.965228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:18.965228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:18.965228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:18.965228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:18.968155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 290/564] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:18.970819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b7e65-511e-4071-8e95-5643bbf82d30 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.971109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b7e65-511e-4071-8e95-5643bbf82d30 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.971380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b7e65-511e-4071-8e95-5643bbf82d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.989333 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-9f7b7e65-511e-4071-8e95-5643bbf82d30 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:18.990541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9f7b7e65-511e-4071-8e95-5643bbf82d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:18.995612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:19.063925 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc252f-6957-4cbc-97b9-13847cd0316c None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:19.073846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 291/565] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:19.073846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1560f-6bcd-4fde-b977-e1dc31b9fc83 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.073846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1560f-6bcd-4fde-b977-e1dc31b9fc83 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.073846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1560f-6bcd-4fde-b977-e1dc31b9fc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.091710 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-9ab1560f-6bcd-4fde-b977-e1dc31b9fc83 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:19.093467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9ab1560f-6bcd-4fde-b977-e1dc31b9fc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:19.116324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021afc3-8454-4755-b92d-6a0732f8774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:19.117083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021afc3-8454-4755-b92d-6a0732f8774d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.117430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021afc3-8454-4755-b92d-6a0732f8774d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.117769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021afc3-8454-4755-b92d-6a0732f8774d None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.118571 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021afc3-8454-4755-b92d-6a0732f8774d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:19.118948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021afc3-8454-4755-b92d-6a0732f8774d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:19.120655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021afc3-8454-4755-b92d-6a0732f8774d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:19.147663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 275/566] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Aug 30 01:03:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:19.148695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021afc3-8454-4755-b92d-6a0732f8774d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:19.188341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 292/567] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:19.189228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1560f-6bcd-4fde-b977-e1dc31b9fc83 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:19.260801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:19.288417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021afc3-8454-4755-b92d-6a0732f8774d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:19.302903 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:19.332843 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08589b78-712a-4e5b-8a8c-07f6f5ae4af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:19.333624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08589b78-712a-4e5b-8a8c-07f6f5ae4af2 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.333998 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08589b78-712a-4e5b-8a8c-07f6f5ae4af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.334360 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08589b78-712a-4e5b-8a8c-07f6f5ae4af2 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.335148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08589b78-712a-4e5b-8a8c-07f6f5ae4af2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:19.335582 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08589b78-712a-4e5b-8a8c-07f6f5ae4af2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:19.337170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08589b78-712a-4e5b-8a8c-07f6f5ae4af2 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:19.339704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1560f-6bcd-4fde-b977-e1dc31b9fc83 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:19.349979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 276/568] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:19.351024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08589b78-712a-4e5b-8a8c-07f6f5ae4af2 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:19.387908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07614e94-90ce-4468-b61d-cd46e76f0a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:19.388632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07614e94-90ce-4468-b61d-cd46e76f0a8d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.388808 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07614e94-90ce-4468-b61d-cd46e76f0a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.389045 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07614e94-90ce-4468-b61d-cd46e76f0a8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.401115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07614e94-90ce-4468-b61d-cd46e76f0a8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:19.402982 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07614e94-90ce-4468-b61d-cd46e76f0a8d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-07614e94-90ce-4468-b61d-cd46e76f0a8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:19.419950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:19.421782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:19.425022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:19.425022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:19.425022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:19.425022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 293/569] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:19.425022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07614e94-90ce-4468-b61d-cd46e76f0a8d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:19.475053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08589b78-712a-4e5b-8a8c-07f6f5ae4af2 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:19.517173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b84b46-2cac-4c86-91a5-3d9ee6abaff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:19.517850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b84b46-2cac-4c86-91a5-3d9ee6abaff7 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.518124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b84b46-2cac-4c86-91a5-3d9ee6abaff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.518431 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b84b46-2cac-4c86-91a5-3d9ee6abaff7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.531325 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b84b46-2cac-4c86-91a5-3d9ee6abaff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-29b84b46-2cac-4c86-91a5-3d9ee6abaff7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:19.537375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:19.539618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 277/570] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:19.541047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b84b46-2cac-4c86-91a5-3d9ee6abaff7 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:19.545542 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:19.553048 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07614e94-90ce-4468-b61d-cd46e76f0a8d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:19.574460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:19.589360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987db65-b717-48d1-ba8c-effca6ede87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:19.589969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987db65-b717-48d1-ba8c-effca6ede87b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.590243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987db65-b717-48d1-ba8c-effca6ede87b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.590506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987db65-b717-48d1-ba8c-effca6ede87b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.591228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987db65-b717-48d1-ba8c-effca6ede87b None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=3330df65b64c4fffbc3bb687d5c51021) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:19.591573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987db65-b717-48d1-ba8c-effca6ede87b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:19.593000 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987db65-b717-48d1-ba8c-effca6ede87b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:19.599022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 294/571] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:19 2022] GET /identity/v3/role_assignments?user.id=3330df65b64c4fffbc3bb687d5c51021&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:19.599845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987db65-b717-48d1-ba8c-effca6ede87b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:19.671941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b84b46-2cac-4c86-91a5-3d9ee6abaff7 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:19.716028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a618049-2bec-41a8-b97b-55c72d8bafba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:19.716682 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a618049-2bec-41a8-b97b-55c72d8bafba None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.716958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a618049-2bec-41a8-b97b-55c72d8bafba None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.717228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a618049-2bec-41a8-b97b-55c72d8bafba None admin] PATH_INFO: `/v3/roles/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.727602 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987db65-b717-48d1-ba8c-effca6ede87b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:19.730443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a618049-2bec-41a8-b97b-55c72d8bafba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-3a618049-2bec-41a8-b97b-55c72d8bafba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:19.735973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:19.737020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 278/572] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:19.737932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a618049-2bec-41a8-b97b-55c72d8bafba None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:19.774046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4cacb-33bd-4665-b588-9b3c23de1644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:19.774943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4cacb-33bd-4665-b588-9b3c23de1644 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.775244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4cacb-33bd-4665-b588-9b3c23de1644 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.775508 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4cacb-33bd-4665-b588-9b3c23de1644 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.776192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4cacb-33bd-4665-b588-9b3c23de1644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:19.776497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4cacb-33bd-4665-b588-9b3c23de1644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:19.777894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4cacb-33bd-4665-b588-9b3c23de1644 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:19.795192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 295/573] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:19.795192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4cacb-33bd-4665-b588-9b3c23de1644 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:19.819118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:19.841826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:19.852367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a618049-2bec-41a8-b97b-55c72d8bafba None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:19.889091 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d3109-f0ed-4792-8ba8-0956ce285764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:19.889734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d3109-f0ed-4792-8ba8-0956ce285764 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.890050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d3109-f0ed-4792-8ba8-0956ce285764 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.890908 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d3109-f0ed-4792-8ba8-0956ce285764 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.902862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d3109-f0ed-4792-8ba8-0956ce285764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-4d0d3109-f0ed-4792-8ba8-0956ce285764 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:19.908313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:19.909251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:19.910078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:19.910078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:19.910078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:19.910078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:19.910078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:19.910078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:19.910078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:19.910078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:19.910078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:19.910078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:19.910078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:19.910078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:19.914186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4cacb-33bd-4665-b588-9b3c23de1644 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:19.914364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 279/574] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:19.915161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d3109-f0ed-4792-8ba8-0956ce285764 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:19.959426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991b81a-935d-46d3-8bd4-ba3aa6b6df9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:19.960060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991b81a-935d-46d3-8bd4-ba3aa6b6df9b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:19.960337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991b81a-935d-46d3-8bd4-ba3aa6b6df9b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:19.960601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991b81a-935d-46d3-8bd4-ba3aa6b6df9b None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:19.961264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991b81a-935d-46d3-8bd4-ba3aa6b6df9b None admin] RBAC: Adding query filter params (name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:19.961559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991b81a-935d-46d3-8bd4-ba3aa6b6df9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:19.963373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991b81a-935d-46d3-8bd4-ba3aa6b6df9b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:19.979252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 296/575] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:19.979252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991b81a-935d-46d3-8bd4-ba3aa6b6df9b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.027771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d3109-f0ed-4792-8ba8-0956ce285764 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.064585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4a925-fe74-4496-8ab5-3fab874f2d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.065188 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4a925-fe74-4496-8ab5-3fab874f2d61 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.065464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4a925-fe74-4496-8ab5-3fab874f2d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.065735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4a925-fe74-4496-8ab5-3fab874f2d61 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.066407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4a925-fe74-4496-8ab5-3fab874f2d61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:20.066707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4a925-fe74-4496-8ab5-3fab874f2d61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.068094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4a925-fe74-4496-8ab5-3fab874f2d61 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.074468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:20.078844 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 280/576] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:20.079662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4a925-fe74-4496-8ab5-3fab874f2d61 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.096394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:20.102999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991b81a-935d-46d3-8bd4-ba3aa6b6df9b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.141782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa1335-dfe8-4142-936c-39a019f986af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.142415 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa1335-dfe8-4142-936c-39a019f986af None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.142691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa1335-dfe8-4142-936c-39a019f986af None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.142958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa1335-dfe8-4142-936c-39a019f986af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.150731 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa1335-dfe8-4142-936c-39a019f986af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.152208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa1335-dfe8-4142-936c-39a019f986af None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-38aa1335-dfe8-4142-936c-39a019f986af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:20.164766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:20.165681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:20.166571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:20.166571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:20.166571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:20.166571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:20.166571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:20.166571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:20.166571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:20.166571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:20.170846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 297/577] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:20.172517 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa1335-dfe8-4142-936c-39a019f986af None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.196735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4a925-fe74-4496-8ab5-3fab874f2d61 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.238295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5c1ba-5615-481c-9827-f5348d42db8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.239360 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5c1ba-5615-481c-9827-f5348d42db8d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.239710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5c1ba-5615-481c-9827-f5348d42db8d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.240073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5c1ba-5615-481c-9827-f5348d42db8d None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.240804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5c1ba-5615-481c-9827-f5348d42db8d None admin] RBAC: Adding query filter params (name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:20.241173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5c1ba-5615-481c-9827-f5348d42db8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.242602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5c1ba-5615-481c-9827-f5348d42db8d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.255888 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 281/578] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:20.255888 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5c1ba-5615-481c-9827-f5348d42db8d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.293654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa1335-dfe8-4142-936c-39a019f986af None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.330831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:20.331330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6a2c5-71f1-47bf-97dd-e5bd2b57f9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.331868 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6a2c5-71f1-47bf-97dd-e5bd2b57f9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.332334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6a2c5-71f1-47bf-97dd-e5bd2b57f9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.332657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6a2c5-71f1-47bf-97dd-e5bd2b57f9b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.345567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6a2c5-71f1-47bf-97dd-e5bd2b57f9b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.347326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6a2c5-71f1-47bf-97dd-e5bd2b57f9b4 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.349478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:20.362116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5c1ba-5615-481c-9827-f5348d42db8d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-51d6a2c5-71f1-47bf-97dd-e5bd2b57f9b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:20.364555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:20.365551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:20.366548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:20.366548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:20.366548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:20.366548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 298/579] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:20.371076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6a2c5-71f1-47bf-97dd-e5bd2b57f9b4 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.402634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915be30-7624-4ff3-801f-671bdbfff1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.403391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915be30-7624-4ff3-801f-671bdbfff1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.403750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915be30-7624-4ff3-801f-671bdbfff1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.404160 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915be30-7624-4ff3-801f-671bdbfff1e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.414926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915be30-7624-4ff3-801f-671bdbfff1e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.416484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915be30-7624-4ff3-801f-671bdbfff1e5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-e915be30-7624-4ff3-801f-671bdbfff1e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:20.432651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:20.433684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:20.435272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:20.435272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:20.435272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:20.435272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 282/580] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:20.435508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915be30-7624-4ff3-801f-671bdbfff1e5 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.487724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6a2c5-71f1-47bf-97dd-e5bd2b57f9b4 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.529388 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b4197-6958-49ab-aeb2-cbdd85d44f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.530124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b4197-6958-49ab-aeb2-cbdd85d44f52 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.530465 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b4197-6958-49ab-aeb2-cbdd85d44f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.531259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b4197-6958-49ab-aeb2-cbdd85d44f52 None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.532241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b4197-6958-49ab-aeb2-cbdd85d44f52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:20.532722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b4197-6958-49ab-aeb2-cbdd85d44f52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.534681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b4197-6958-49ab-aeb2-cbdd85d44f52 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.549589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 299/581] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:20.550433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b4197-6958-49ab-aeb2-cbdd85d44f52 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.557905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915be30-7624-4ff3-801f-671bdbfff1e5 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.588137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:20.599740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99124da1-f19a-4b70-a327-a214dec0c766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.600424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99124da1-f19a-4b70-a327-a214dec0c766 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.600698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99124da1-f19a-4b70-a327-a214dec0c766 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.600963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99124da1-f19a-4b70-a327-a214dec0c766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.614718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99124da1-f19a-4b70-a327-a214dec0c766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.616191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99124da1-f19a-4b70-a327-a214dec0c766 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.623025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-99124da1-f19a-4b70-a327-a214dec0c766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:20.630728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:20.631665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:20.632608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:20.632608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:20.632608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:20.632608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 283/582] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:20.632831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99124da1-f19a-4b70-a327-a214dec0c766 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.680909 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b4197-6958-49ab-aeb2-cbdd85d44f52 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.721446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b2bcb-002d-40d4-9471-50421cc7811f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.722073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b2bcb-002d-40d4-9471-50421cc7811f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.722455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b2bcb-002d-40d4-9471-50421cc7811f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.722725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b2bcb-002d-40d4-9471-50421cc7811f None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.723395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b2bcb-002d-40d4-9471-50421cc7811f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:20.723695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b2bcb-002d-40d4-9471-50421cc7811f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.725165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b2bcb-002d-40d4-9471-50421cc7811f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.737917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 300/583] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:20.738639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b2bcb-002d-40d4-9471-50421cc7811f None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.763028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99124da1-f19a-4b70-a327-a214dec0c766 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.800235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4464d-7e6b-4c51-b9f7-8bbad6c1914f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.800782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4464d-7e6b-4c51-b9f7-8bbad6c1914f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.801042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4464d-7e6b-4c51-b9f7-8bbad6c1914f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.801316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4464d-7e6b-4c51-b9f7-8bbad6c1914f None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.801962 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4464d-7e6b-4c51-b9f7-8bbad6c1914f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:20.802263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4464d-7e6b-4c51-b9f7-8bbad6c1914f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.803626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4464d-7e6b-4c51-b9f7-8bbad6c1914f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.817899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 284/584] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:20.819072 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4464d-7e6b-4c51-b9f7-8bbad6c1914f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.849633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b2bcb-002d-40d4-9471-50421cc7811f None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:20.865324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:20.866710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:20.888286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2ddc4-8c00-4212-9f20-7ed2a09b0570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:20.888950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2ddc4-8c00-4212-9f20-7ed2a09b0570 None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.889229 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2ddc4-8c00-4212-9f20-7ed2a09b0570 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.889502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2ddc4-8c00-4212-9f20-7ed2a09b0570 None admin] PATH_INFO: `/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/049399a752ed4646a13f8e4a3a62d8e9/roles/d3cef78b3402418ea77f177cc74fecb1` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.906153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2ddc4-8c00-4212-9f20-7ed2a09b0570 None admin] RBAC: Authorizing `identity:create_grant(project_id=9188023bf51d4737b801e3fbd9af95b1, user_id=049399a752ed4646a13f8e4a3a62d8e9, role_id=d3cef78b3402418ea77f177cc74fecb1)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:20.908024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2ddc4-8c00-4212-9f20-7ed2a09b0570 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:20.932028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 301/585] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:03:20 2022] PUT /identity/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/049399a752ed4646a13f8e4a3a62d8e9/roles/d3cef78b3402418ea77f177cc74fecb1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:20.932028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2ddc4-8c00-4212-9f20-7ed2a09b0570 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:20.952215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4464d-7e6b-4c51-b9f7-8bbad6c1914f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.010963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a31b18-1ab5-4d50-802d-3718f432cc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.012053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a31b18-1ab5-4d50-802d-3718f432cc64 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.012441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a31b18-1ab5-4d50-802d-3718f432cc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.013186 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a31b18-1ab5-4d50-802d-3718f432cc64 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.014170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a31b18-1ab5-4d50-802d-3718f432cc64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:21.014480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a31b18-1ab5-4d50-802d-3718f432cc64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.016669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a31b18-1ab5-4d50-802d-3718f432cc64 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:21.038028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 285/586] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:21.039114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a31b18-1ab5-4d50-802d-3718f432cc64 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:21.079567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2ddc4-8c00-4212-9f20-7ed2a09b0570 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.127728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1db418-07d5-4fe9-a2a4-37fa7da0912d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.128473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1db418-07d5-4fe9-a2a4-37fa7da0912d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.128750 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1db418-07d5-4fe9-a2a4-37fa7da0912d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.129018 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1db418-07d5-4fe9-a2a4-37fa7da0912d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.139606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1db418-07d5-4fe9-a2a4-37fa7da0912d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.142224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1db418-07d5-4fe9-a2a4-37fa7da0912d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:21.160604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-db1db418-07d5-4fe9-a2a4-37fa7da0912d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:21.162562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:21.163526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:21.164647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:21.164647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:21.164647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:21.164647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:21.164647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:21.164647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:21.164647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:21.164647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:21.164647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 01:03:21.164647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:21.164647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 302/587] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:21.165222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1db418-07d5-4fe9-a2a4-37fa7da0912d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:21.172820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a31b18-1ab5-4d50-802d-3718f432cc64 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.204748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:21.215070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f808615-0439-4cbd-971c-f11a5a512ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.216103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f808615-0439-4cbd-971c-f11a5a512ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.216580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f808615-0439-4cbd-971c-f11a5a512ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.217083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f808615-0439-4cbd-971c-f11a5a512ea0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.227094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f808615-0439-4cbd-971c-f11a5a512ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.229112 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f808615-0439-4cbd-971c-f11a5a512ea0 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-5f808615-0439-4cbd-971c-f11a5a512ea0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:21.242719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:21.243681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:21.244969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:21.244969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:21.244969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:21.244969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:21.244969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:21.244969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:21.244969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:21.244969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:21.244969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 01:03:21.244969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:21.247470 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 286/588] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:21.248601 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f808615-0439-4cbd-971c-f11a5a512ea0 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:21.302592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1db418-07d5-4fe9-a2a4-37fa7da0912d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.340024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87754cfd-8764-4738-9a12-dd2a3fef40d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.340547 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87754cfd-8764-4738-9a12-dd2a3fef40d8 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.340826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87754cfd-8764-4738-9a12-dd2a3fef40d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.341096 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87754cfd-8764-4738-9a12-dd2a3fef40d8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.349940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87754cfd-8764-4738-9a12-dd2a3fef40d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.351434 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87754cfd-8764-4738-9a12-dd2a3fef40d8 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-87754cfd-8764-4738-9a12-dd2a3fef40d8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:21.363406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:21.364493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:21.365403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:21.365403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:21.365403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:21.365403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:21.365403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:21.365403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:21.365403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:21.365403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:21.365403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:03:21.365403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:21.369348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 303/589] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:21.370232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87754cfd-8764-4738-9a12-dd2a3fef40d8 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:21.371520 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f808615-0439-4cbd-971c-f11a5a512ea0 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.411312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d6aa3-c3df-4d14-8119-ea097709a232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.412335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d6aa3-c3df-4d14-8119-ea097709a232 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.412834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d6aa3-c3df-4d14-8119-ea097709a232 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.413383 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d6aa3-c3df-4d14-8119-ea097709a232 None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.414365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d6aa3-c3df-4d14-8119-ea097709a232 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:21.414871 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d6aa3-c3df-4d14-8119-ea097709a232 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.416736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d6aa3-c3df-4d14-8119-ea097709a232 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:21.436186 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 287/590] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Aug 30 01:03:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:21.438317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d6aa3-c3df-4d14-8119-ea097709a232 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:21.459023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:21.470679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:21.489862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87754cfd-8764-4738-9a12-dd2a3fef40d8 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.533768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035ce58-a2bc-49ab-9662-bb9da27182fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.534504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035ce58-a2bc-49ab-9662-bb9da27182fb None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.534830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035ce58-a2bc-49ab-9662-bb9da27182fb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.535146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035ce58-a2bc-49ab-9662-bb9da27182fb None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.536159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035ce58-a2bc-49ab-9662-bb9da27182fb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:21.536512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035ce58-a2bc-49ab-9662-bb9da27182fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.538174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035ce58-a2bc-49ab-9662-bb9da27182fb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:21.553456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d6aa3-c3df-4d14-8119-ea097709a232 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.563406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 304/591] 10.222.0.44 () {56 vars in 1139 bytes} [Tue Aug 30 01:03:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:21.564273 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035ce58-a2bc-49ab-9662-bb9da27182fb None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:21.595916 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08c17d-70a1-4d3e-b5b0-e804542e57ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.597087 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08c17d-70a1-4d3e-b5b0-e804542e57ed None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.597642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08c17d-70a1-4d3e-b5b0-e804542e57ed None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.598144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08c17d-70a1-4d3e-b5b0-e804542e57ed None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.599329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08c17d-70a1-4d3e-b5b0-e804542e57ed None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:21.599890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08c17d-70a1-4d3e-b5b0-e804542e57ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.601685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08c17d-70a1-4d3e-b5b0-e804542e57ed None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:21.621740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 288/592] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Aug 30 01:03:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:21.623056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08c17d-70a1-4d3e-b5b0-e804542e57ed None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:21.679878 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035ce58-a2bc-49ab-9662-bb9da27182fb None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.714590 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:21.719327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771308fb-4ea7-41ed-bafc-a64c408be7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.719966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771308fb-4ea7-41ed-bafc-a64c408be7db None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.720272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771308fb-4ea7-41ed-bafc-a64c408be7db None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.720541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771308fb-4ea7-41ed-bafc-a64c408be7db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.731448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771308fb-4ea7-41ed-bafc-a64c408be7db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.732921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771308fb-4ea7-41ed-bafc-a64c408be7db None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:21.736095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:21.737832 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08c17d-70a1-4d3e-b5b0-e804542e57ed None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-771308fb-4ea7-41ed-bafc-a64c408be7db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:21.751553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:21.752586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:21.753440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:21.753440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:21.753440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:21.753440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 305/593] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:21.754243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f87b7-eed5-4256-a213-266c6a9183c2 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.754243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f87b7-eed5-4256-a213-266c6a9183c2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.754402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f87b7-eed5-4256-a213-266c6a9183c2 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.759743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 306/594] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:21.760561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f87b7-eed5-4256-a213-266c6a9183c2 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:21.784494 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d425a5-063c-49b5-976b-036469f387f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.785604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d425a5-063c-49b5-976b-036469f387f0 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.786269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d425a5-063c-49b5-976b-036469f387f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.786706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d425a5-063c-49b5-976b-036469f387f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.799453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d425a5-063c-49b5-976b-036469f387f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.800956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d425a5-063c-49b5-976b-036469f387f0 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-a9d425a5-063c-49b5-976b-036469f387f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:21.816125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:21.817092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:21.817946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:21.817946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:21.817946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:21.822162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 289/595] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:21.822918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d425a5-063c-49b5-976b-036469f387f0 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:21.872621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f87b7-eed5-4256-a213-266c6a9183c2 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.911751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c8ca5-677d-4006-a2ed-d7e1919c8ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.912403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c8ca5-677d-4006-a2ed-d7e1919c8ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.912718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c8ca5-677d-4006-a2ed-d7e1919c8ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.912933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c8ca5-677d-4006-a2ed-d7e1919c8ad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.924918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c8ca5-677d-4006-a2ed-d7e1919c8ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.926378 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c8ca5-677d-4006-a2ed-d7e1919c8ad6 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:21.940853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d425a5-063c-49b5-976b-036469f387f0 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-359c8ca5-677d-4006-a2ed-d7e1919c8ad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:21.942657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:21.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:21.945228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:21.945228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:21.945228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:21.945228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 307/596] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:21.945228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1d7ad-62da-4261-b24f-7b6b0760f329 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.945528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1d7ad-62da-4261-b24f-7b6b0760f329 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.945598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1d7ad-62da-4261-b24f-7b6b0760f329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.965121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e9b1d7ad-62da-4261-b24f-7b6b0760f329 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:21.966313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e9b1d7ad-62da-4261-b24f-7b6b0760f329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:21.974318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:21.987066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432af31c-5497-4b66-9e45-d5c054658c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:21.987974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432af31c-5497-4b66-9e45-d5c054658c0a None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:21.988281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432af31c-5497-4b66-9e45-d5c054658c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:21.988547 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432af31c-5497-4b66-9e45-d5c054658c0a None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:21.989429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432af31c-5497-4b66-9e45-d5c054658c0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:21.990087 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432af31c-5497-4b66-9e45-d5c054658c0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:21.991404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432af31c-5497-4b66-9e45-d5c054658c0a None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:22.005051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 290/597] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:22.006193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432af31c-5497-4b66-9e45-d5c054658c0a None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:22.024256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:22.053544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 308/598] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:22.061465 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1d7ad-62da-4261-b24f-7b6b0760f329 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:22.139407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432af31c-5497-4b66-9e45-d5c054658c0a None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:22.181831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71428b93-7b8b-44a1-b44f-ad556d7df36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:22.182443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71428b93-7b8b-44a1-b44f-ad556d7df36c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:22.182720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71428b93-7b8b-44a1-b44f-ad556d7df36c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:22.182983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71428b93-7b8b-44a1-b44f-ad556d7df36c None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:22.184070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71428b93-7b8b-44a1-b44f-ad556d7df36c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:22.184070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71428b93-7b8b-44a1-b44f-ad556d7df36c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:22.185736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71428b93-7b8b-44a1-b44f-ad556d7df36c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:22.199330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 291/599] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:22.200151 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1d7ad-62da-4261-b24f-7b6b0760f329 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:22.200265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71428b93-7b8b-44a1-b44f-ad556d7df36c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:22.248999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b125b19-fd91-436e-a4e4-a557ce6b6eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:22.250895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b125b19-fd91-436e-a4e4-a557ce6b6eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:22.251741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b125b19-fd91-436e-a4e4-a557ce6b6eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:22.252234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b125b19-fd91-436e-a4e4-a557ce6b6eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:22.253356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b125b19-fd91-436e-a4e4-a557ce6b6eb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:22.254475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:22.255084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b125b19-fd91-436e-a4e4-a557ce6b6eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:22.257436 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b125b19-fd91-436e-a4e4-a557ce6b6eb7 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:22.276202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 309/600] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:22.277667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b125b19-fd91-436e-a4e4-a557ce6b6eb7 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:22.320791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71428b93-7b8b-44a1-b44f-ad556d7df36c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:22.325967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:22.357790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b342d1a-9ab4-498d-a663-b36b6c14c02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:22.358425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b342d1a-9ab4-498d-a663-b36b6c14c02d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:22.358761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b342d1a-9ab4-498d-a663-b36b6c14c02d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:22.359109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b342d1a-9ab4-498d-a663-b36b6c14c02d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:22.366417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b342d1a-9ab4-498d-a663-b36b6c14c02d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:22.367861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b342d1a-9ab4-498d-a663-b36b6c14c02d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-9b342d1a-9ab4-498d-a663-b36b6c14c02d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:22.380364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:22.381730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:22.382602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:22.382602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:22.382602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:22.382602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:22.382602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:22.382602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:22.382602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:22.382602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:22.382602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 292/601] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:22.382602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b342d1a-9ab4-498d-a663-b36b6c14c02d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:22.415418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b125b19-fd91-436e-a4e4-a557ce6b6eb7 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:22.460847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb5660-99df-47dc-a158-db3d3c6845f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:22.461492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb5660-99df-47dc-a158-db3d3c6845f0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:22.461767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb5660-99df-47dc-a158-db3d3c6845f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:22.462032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb5660-99df-47dc-a158-db3d3c6845f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:22.470529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb5660-99df-47dc-a158-db3d3c6845f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:22.472053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb5660-99df-47dc-a158-db3d3c6845f0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-49cb5660-99df-47dc-a158-db3d3c6845f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:22.485086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:22.486154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:22.487124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:22.487124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:22.487124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:22.487124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:22.487124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:22.487124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:22.487124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:22.487124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:22.487124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 310/602] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:22.487124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb5660-99df-47dc-a158-db3d3c6845f0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:22.498907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b342d1a-9ab4-498d-a663-b36b6c14c02d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:22.508081 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:22.540228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b960b9-377c-4dce-a617-dd216a26483b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:22.540976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b960b9-377c-4dce-a617-dd216a26483b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:22.541259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b960b9-377c-4dce-a617-dd216a26483b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:22.541529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b960b9-377c-4dce-a617-dd216a26483b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:22.550948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b960b9-377c-4dce-a617-dd216a26483b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:22.552452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b960b9-377c-4dce-a617-dd216a26483b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-a6b960b9-377c-4dce-a617-dd216a26483b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:22.565301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:22.566245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:22.567148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:22.567148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:22.567148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:22.567148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:22.567148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:22.567148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:22.567148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:22.567148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:22.567148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 293/603] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:22.567927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b960b9-377c-4dce-a617-dd216a26483b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:22.619144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:22.627943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb5660-99df-47dc-a158-db3d3c6845f0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:22.667098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520deefd-f662-4a37-8447-e2ede230d8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:22.667651 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520deefd-f662-4a37-8447-e2ede230d8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:22.667923 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520deefd-f662-4a37-8447-e2ede230d8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:22.668322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520deefd-f662-4a37-8447-e2ede230d8e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:22.681131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520deefd-f662-4a37-8447-e2ede230d8e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:22.689271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b960b9-377c-4dce-a617-dd216a26483b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-520deefd-f662-4a37-8447-e2ede230d8e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:22.689445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:22.690381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 311/604] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:22.691222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520deefd-f662-4a37-8447-e2ede230d8e6 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:22.731331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f3e9f-16fa-495e-83a9-ef2de26487ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:22.731947 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f3e9f-16fa-495e-83a9-ef2de26487ee None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:22.732262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f3e9f-16fa-495e-83a9-ef2de26487ee None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:22.732540 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f3e9f-16fa-495e-83a9-ef2de26487ee None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:22.733240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f3e9f-16fa-495e-83a9-ef2de26487ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:22.733550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f3e9f-16fa-495e-83a9-ef2de26487ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:22.735586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f3e9f-16fa-495e-83a9-ef2de26487ee None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:22.754610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 294/605] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:22.754610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f3e9f-16fa-495e-83a9-ef2de26487ee None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:22.785084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:22.823296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520deefd-f662-4a37-8447-e2ede230d8e6 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:22.865871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da06094-5cd8-44b9-a5d0-3b8e3b9a261c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:22.866494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da06094-5cd8-44b9-a5d0-3b8e3b9a261c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:22.866770 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da06094-5cd8-44b9-a5d0-3b8e3b9a261c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:22.867049 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da06094-5cd8-44b9-a5d0-3b8e3b9a261c None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:22.867803 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da06094-5cd8-44b9-a5d0-3b8e3b9a261c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:22.868126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da06094-5cd8-44b9-a5d0-3b8e3b9a261c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:22.869576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da06094-5cd8-44b9-a5d0-3b8e3b9a261c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:22.874852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f3e9f-16fa-495e-83a9-ef2de26487ee None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:22.883647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 312/606] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:22.884594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da06094-5cd8-44b9-a5d0-3b8e3b9a261c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:22.903299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:22.926329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffa76f-1dbb-4a3c-9028-41dbc235d744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:22.926945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffa76f-1dbb-4a3c-9028-41dbc235d744 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:22.927233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffa76f-1dbb-4a3c-9028-41dbc235d744 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:22.927509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffa76f-1dbb-4a3c-9028-41dbc235d744 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:22.928254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffa76f-1dbb-4a3c-9028-41dbc235d744 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:22.928544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffa76f-1dbb-4a3c-9028-41dbc235d744 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:22.930049 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffa76f-1dbb-4a3c-9028-41dbc235d744 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:22.942467 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 295/607] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:22.943282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffa76f-1dbb-4a3c-9028-41dbc235d744 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:23.006613 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da06094-5cd8-44b9-a5d0-3b8e3b9a261c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:23.043445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7434db-cfcb-4f78-85f3-5be172137e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:23.044075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7434db-cfcb-4f78-85f3-5be172137e01 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:23.044355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7434db-cfcb-4f78-85f3-5be172137e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:23.044875 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7434db-cfcb-4f78-85f3-5be172137e01 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:23.045569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7434db-cfcb-4f78-85f3-5be172137e01 None admin] RBAC: Adding query filter params (name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:23.045869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7434db-cfcb-4f78-85f3-5be172137e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:23.047868 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7434db-cfcb-4f78-85f3-5be172137e01 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:23.047979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:23.056617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 313/608] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:23.057418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7434db-cfcb-4f78-85f3-5be172137e01 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:23.069386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffa76f-1dbb-4a3c-9028-41dbc235d744 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:23.105482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23c73d-2641-4c58-9612-5cd7b5a97eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:23.106145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23c73d-2641-4c58-9612-5cd7b5a97eaf None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:23.106424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23c73d-2641-4c58-9612-5cd7b5a97eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:23.106695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23c73d-2641-4c58-9612-5cd7b5a97eaf None admin] PATH_INFO: `/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/92c9d444b3294ed18c3d488cf0a3eeb2/roles/d3cef78b3402418ea77f177cc74fecb1` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:23.122309 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23c73d-2641-4c58-9612-5cd7b5a97eaf None admin] RBAC: Authorizing `identity:create_grant(project_id=9188023bf51d4737b801e3fbd9af95b1, user_id=92c9d444b3294ed18c3d488cf0a3eeb2, role_id=d3cef78b3402418ea77f177cc74fecb1)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:23.123741 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23c73d-2641-4c58-9612-5cd7b5a97eaf None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:23.136966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 296/609] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:03:22 2022] PUT /identity/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/92c9d444b3294ed18c3d488cf0a3eeb2/roles/d3cef78b3402418ea77f177cc74fecb1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:23.137766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23c73d-2641-4c58-9612-5cd7b5a97eaf None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:23.137902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:23.165247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7434db-cfcb-4f78-85f3-5be172137e01 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:23.206976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c758e-2444-4771-8c89-2ce07184d2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:23.208064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c758e-2444-4771-8c89-2ce07184d2b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:23.208516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c758e-2444-4771-8c89-2ce07184d2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:23.208948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c758e-2444-4771-8c89-2ce07184d2b3 None admin] PATH_INFO: `/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/86cea2f9238b42e0a7cb26bac494a8c7/roles/d3cef78b3402418ea77f177cc74fecb1` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:23.229874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c758e-2444-4771-8c89-2ce07184d2b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9188023bf51d4737b801e3fbd9af95b1, user_id=86cea2f9238b42e0a7cb26bac494a8c7, role_id=d3cef78b3402418ea77f177cc74fecb1)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:23.231381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c758e-2444-4771-8c89-2ce07184d2b3 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:23.250101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 314/610] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:03:23 2022] PUT /identity/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/86cea2f9238b42e0a7cb26bac494a8c7/roles/d3cef78b3402418ea77f177cc74fecb1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:23.254428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456c2d4-cf25-4921-b28d-9e6886d93255 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:23.255077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456c2d4-cf25-4921-b28d-9e6886d93255 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:23.255824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456c2d4-cf25-4921-b28d-9e6886d93255 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:23.263140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 315/611] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:23.263140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456c2d4-cf25-4921-b28d-9e6886d93255 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:23.291176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23c73d-2641-4c58-9612-5cd7b5a97eaf None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:23.339817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51720dac-88d3-49fa-b27a-576d1a63e7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:23.340935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51720dac-88d3-49fa-b27a-576d1a63e7bd None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:23.341455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51720dac-88d3-49fa-b27a-576d1a63e7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:23.341951 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51720dac-88d3-49fa-b27a-576d1a63e7bd None admin] PATH_INFO: `/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/534da5b47dda43dabaab2cb6aa683342/roles/d3cef78b3402418ea77f177cc74fecb1` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:23.346923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:23.367611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51720dac-88d3-49fa-b27a-576d1a63e7bd None admin] RBAC: Authorizing `identity:create_grant(project_id=9188023bf51d4737b801e3fbd9af95b1, user_id=534da5b47dda43dabaab2cb6aa683342, role_id=d3cef78b3402418ea77f177cc74fecb1)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:23.370880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51720dac-88d3-49fa-b27a-576d1a63e7bd None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:23.392639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 297/612] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:03:23 2022] PUT /identity/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/534da5b47dda43dabaab2cb6aa683342/roles/d3cef78b3402418ea77f177cc74fecb1 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:23.396370 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835902d-6915-4f14-9a83-497c51bd25a9 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:23.396876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835902d-6915-4f14-9a83-497c51bd25a9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:23.397305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835902d-6915-4f14-9a83-497c51bd25a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:23.421834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-5835902d-6915-4f14-9a83-497c51bd25a9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:23.423359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5835902d-6915-4f14-9a83-497c51bd25a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:23.440996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456c2d4-cf25-4921-b28d-9e6886d93255 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:23.482685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:23.495917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534f9cb-1687-4d43-a799-8fc2c8ba9a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:23.497188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534f9cb-1687-4d43-a799-8fc2c8ba9a1e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:23.497683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534f9cb-1687-4d43-a799-8fc2c8ba9a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:23.498173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534f9cb-1687-4d43-a799-8fc2c8ba9a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:23.513031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534f9cb-1687-4d43-a799-8fc2c8ba9a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:23.516495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534f9cb-1687-4d43-a799-8fc2c8ba9a1e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:23.536061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 298/613] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-7534f9cb-1687-4d43-a799-8fc2c8ba9a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:23.544061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:23.545943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:23.547729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:23.547729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:23.547729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:23.547729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 316/614] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:23.551607 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835902d-6915-4f14-9a83-497c51bd25a9 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:23.673684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835902d-6915-4f14-9a83-497c51bd25a9 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:23.693702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:23.716933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3bb06-6f69-4382-bfd9-859fa58d915b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:23.717675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3bb06-6f69-4382-bfd9-859fa58d915b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:23.718583 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3bb06-6f69-4382-bfd9-859fa58d915b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:23.718946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3bb06-6f69-4382-bfd9-859fa58d915b None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:23.719711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3bb06-6f69-4382-bfd9-859fa58d915b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:23.720141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3bb06-6f69-4382-bfd9-859fa58d915b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:23.721641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3bb06-6f69-4382-bfd9-859fa58d915b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:23.738900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 299/615] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:23.744054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534f9cb-1687-4d43-a799-8fc2c8ba9a1e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:23.877539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534f9cb-1687-4d43-a799-8fc2c8ba9a1e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:23.885040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3bb06-6f69-4382-bfd9-859fa58d915b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:23.934433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08890c2-ef71-4bf3-9027-c8b6eef5e730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:23.935131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08890c2-ef71-4bf3-9027-c8b6eef5e730 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:23.935407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08890c2-ef71-4bf3-9027-c8b6eef5e730 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:23.935673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08890c2-ef71-4bf3-9027-c8b6eef5e730 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:23.946905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08890c2-ef71-4bf3-9027-c8b6eef5e730 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:23.948713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08890c2-ef71-4bf3-9027-c8b6eef5e730 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-d08890c2-ef71-4bf3-9027-c8b6eef5e730 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:23.963306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:23.964360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:23.965282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:23.965282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:23.965282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:23.965282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:23.965282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:23.965282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:23.965282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:23.965282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:23.965282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:03:23.965282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:23.967532 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 317/616] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:23.978491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08890c2-ef71-4bf3-9027-c8b6eef5e730 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:24.050324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3bb06-6f69-4382-bfd9-859fa58d915b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:24.064344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:24.112021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31464b07-b17b-4d62-b006-9990e68a08f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:24.112021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31464b07-b17b-4d62-b006-9990e68a08f2 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:24.112021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31464b07-b17b-4d62-b006-9990e68a08f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:24.112021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31464b07-b17b-4d62-b006-9990e68a08f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:24.126434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31464b07-b17b-4d62-b006-9990e68a08f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-31464b07-b17b-4d62-b006-9990e68a08f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:24.133686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:24.134853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:24.135889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 300/617] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:24.142689 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31464b07-b17b-4d62-b006-9990e68a08f2 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:24.191051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08890c2-ef71-4bf3-9027-c8b6eef5e730 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:24.204662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:24.287797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31464b07-b17b-4d62-b006-9990e68a08f2 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:24.299160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a73cb-fb83-460c-9a03-fae38fca0ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:24.299794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a73cb-fb83-460c-9a03-fae38fca0ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:24.300099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a73cb-fb83-460c-9a03-fae38fca0ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:24.300432 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a73cb-fb83-460c-9a03-fae38fca0ee0 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:24.301192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a73cb-fb83-460c-9a03-fae38fca0ee0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:24.301537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a73cb-fb83-460c-9a03-fae38fca0ee0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:24.303066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a73cb-fb83-460c-9a03-fae38fca0ee0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:24.333686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69ec5c-0c9c-4044-a543-7ba4664f61c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:24.334327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69ec5c-0c9c-4044-a543-7ba4664f61c3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:24.334610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69ec5c-0c9c-4044-a543-7ba4664f61c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:24.334887 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69ec5c-0c9c-4044-a543-7ba4664f61c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:24.335580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69ec5c-0c9c-4044-a543-7ba4664f61c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:24.335889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69ec5c-0c9c-4044-a543-7ba4664f61c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:24.337361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69ec5c-0c9c-4044-a543-7ba4664f61c3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:24.349935 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 318/619] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Aug 30 01:03:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:24.353220 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 301/619] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:24.354616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a73cb-fb83-460c-9a03-fae38fca0ee0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:24.361128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69ec5c-0c9c-4044-a543-7ba4664f61c3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:24.548316 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:24.551915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a73cb-fb83-460c-9a03-fae38fca0ee0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:24.572330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:24.582642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69ec5c-0c9c-4044-a543-7ba4664f61c3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:24.622653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca34a1-de36-4e95-b909-97a0dcc2267f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:24.623295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca34a1-de36-4e95-b909-97a0dcc2267f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:24.623566 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca34a1-de36-4e95-b909-97a0dcc2267f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:24.623829 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca34a1-de36-4e95-b909-97a0dcc2267f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:24.643528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca34a1-de36-4e95-b909-97a0dcc2267f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:24.646765 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca34a1-de36-4e95-b909-97a0dcc2267f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:24.647760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e258a-8e8d-44c2-8d55-e13b6fc0ea6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:24.648421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e258a-8e8d-44c2-8d55-e13b6fc0ea6b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:24.648690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e258a-8e8d-44c2-8d55-e13b6fc0ea6b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:24.648955 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e258a-8e8d-44c2-8d55-e13b6fc0ea6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:24.670176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e258a-8e8d-44c2-8d55-e13b6fc0ea6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:24.675674 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e258a-8e8d-44c2-8d55-e13b6fc0ea6b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-ccca34a1-de36-4e95-b909-97a0dcc2267f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:24.678549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:24.679521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:24.680464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:24.680464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:24.680464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:24.682803 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 319/620] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:24.685803 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca34a1-de36-4e95-b909-97a0dcc2267f None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-6a6e258a-8e8d-44c2-8d55-e13b6fc0ea6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:24.711318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:24.712343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:24.713216 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:24.713216 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:24.713216 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:24.713804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 302/621] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:24.718175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e258a-8e8d-44c2-8d55-e13b6fc0ea6b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:24.880526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca34a1-de36-4e95-b909-97a0dcc2267f None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:24.898845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e258a-8e8d-44c2-8d55-e13b6fc0ea6b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:24.950926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19750d-ef93-4f01-996d-a40e409a6c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:24.951548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19750d-ef93-4f01-996d-a40e409a6c7c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:24.951825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19750d-ef93-4f01-996d-a40e409a6c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:24.952119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19750d-ef93-4f01-996d-a40e409a6c7c None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:24.952866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19750d-ef93-4f01-996d-a40e409a6c7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:24.953152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19750d-ef93-4f01-996d-a40e409a6c7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:24.954587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19750d-ef93-4f01-996d-a40e409a6c7c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:24.967752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 320/622] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:24.973035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edf311-9731-4436-b350-14444c98d646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:24.973212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19750d-ef93-4f01-996d-a40e409a6c7c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:24.973773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edf311-9731-4436-b350-14444c98d646 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:24.974060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edf311-9731-4436-b350-14444c98d646 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:24.974341 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edf311-9731-4436-b350-14444c98d646 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:24.975002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2edf311-9731-4436-b350-14444c98d646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:24.975309 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2edf311-9731-4436-b350-14444c98d646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:24.985011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2edf311-9731-4436-b350-14444c98d646 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:25.000384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:25.010607 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 303/623] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:25.017352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edf311-9731-4436-b350-14444c98d646 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:25.042804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:25.161063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19750d-ef93-4f01-996d-a40e409a6c7c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:25.203927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edf311-9731-4436-b350-14444c98d646 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:25.225263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99df65-2d34-47ab-b78c-8fe01574a325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:25.225902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99df65-2d34-47ab-b78c-8fe01574a325 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.226172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99df65-2d34-47ab-b78c-8fe01574a325 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.226433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99df65-2d34-47ab-b78c-8fe01574a325 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.239882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99df65-2d34-47ab-b78c-8fe01574a325 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:25.244268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99df65-2d34-47ab-b78c-8fe01574a325 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-1a99df65-2d34-47ab-b78c-8fe01574a325 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:25.259011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:25.260017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:25.260967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:25.260967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:25.260967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:25.260967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:25.260967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:25.260967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:25.260967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:25.260967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:25.260967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:03:25.260967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:25.260967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 321/624] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Aug 30 01:03:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:25.266863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99df65-2d34-47ab-b78c-8fe01574a325 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:25.288732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ae7ef-a1e3-4106-a856-9fbe5e984381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:25.289831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ae7ef-a1e3-4106-a856-9fbe5e984381 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.290348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ae7ef-a1e3-4106-a856-9fbe5e984381 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.290820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ae7ef-a1e3-4106-a856-9fbe5e984381 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.305483 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ae7ef-a1e3-4106-a856-9fbe5e984381 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:25.307270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ae7ef-a1e3-4106-a856-9fbe5e984381 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-9a6ae7ef-a1e3-4106-a856-9fbe5e984381 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:25.329990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:25.330958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:25.331842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:25.331842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:25.331842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:25.331842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:25.331842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:25.331842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:25.331842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:25.331842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:25.341694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 304/625] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:25.348835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ae7ef-a1e3-4106-a856-9fbe5e984381 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:25.369995 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:25.380433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99df65-2d34-47ab-b78c-8fe01574a325 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:25.449686 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27caad2-5ef1-41a4-927a-33dfb4b60719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:25.451370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27caad2-5ef1-41a4-927a-33dfb4b60719 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.452049 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27caad2-5ef1-41a4-927a-33dfb4b60719 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.454487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27caad2-5ef1-41a4-927a-33dfb4b60719 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.455461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27caad2-5ef1-41a4-927a-33dfb4b60719 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:25.456199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27caad2-5ef1-41a4-927a-33dfb4b60719 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:25.458720 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27caad2-5ef1-41a4-927a-33dfb4b60719 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:25.495503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 322/626] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Aug 30 01:03:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:25.505362 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27caad2-5ef1-41a4-927a-33dfb4b60719 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:25.513791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:25.547170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ae7ef-a1e3-4106-a856-9fbe5e984381 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:25.578530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc2bb8-4166-492a-bb39-7dbbcd4e383b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:25.579231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2bb8-4166-492a-bb39-7dbbcd4e383b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.579478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2bb8-4166-492a-bb39-7dbbcd4e383b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.579676 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2bb8-4166-492a-bb39-7dbbcd4e383b None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.580395 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc2bb8-4166-492a-bb39-7dbbcd4e383b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:25.580702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc2bb8-4166-492a-bb39-7dbbcd4e383b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:25.582125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc2bb8-4166-492a-bb39-7dbbcd4e383b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:25.595326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 305/627] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:25.596643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c852d1-fdb4-4d37-88f4-54fe6298cb26 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.596932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c852d1-fdb4-4d37-88f4-54fe6298cb26 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.597260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c852d1-fdb4-4d37-88f4-54fe6298cb26 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.598785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 306/628] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:25.599919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f249b68-4923-4cf8-8787-89f20b682b5a None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.600226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f249b68-4923-4cf8-8787-89f20b682b5a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.600555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f249b68-4923-4cf8-8787-89f20b682b5a None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.601990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 307/629] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:25.605397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7a4dc-984b-4498-928c-c2f537012799 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.605674 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7a4dc-984b-4498-928c-c2f537012799 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.605952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7a4dc-984b-4498-928c-c2f537012799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.629463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-37b7a4dc-984b-4498-928c-c2f537012799 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:25.631251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-37b7a4dc-984b-4498-928c-c2f537012799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:25.660137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27caad2-5ef1-41a4-927a-33dfb4b60719 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:25.696292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:25.706272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f266495-e0fb-42a8-a36b-94b82f51044e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:25.706812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f266495-e0fb-42a8-a36b-94b82f51044e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.708109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f266495-e0fb-42a8-a36b-94b82f51044e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.708109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f266495-e0fb-42a8-a36b-94b82f51044e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.724147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 308/630] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:25.725583 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7a4dc-984b-4498-928c-c2f537012799 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:25.784662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f266495-e0fb-42a8-a36b-94b82f51044e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:25.786535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f266495-e0fb-42a8-a36b-94b82f51044e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:25.807306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:25.854561 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7a4dc-984b-4498-928c-c2f537012799 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:25.891673 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f5bd1-2ee5-463e-b46a-05b02e3e8aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:25.892682 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f5bd1-2ee5-463e-b46a-05b02e3e8aef None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.893142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f5bd1-2ee5-463e-b46a-05b02e3e8aef None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.893617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f5bd1-2ee5-463e-b46a-05b02e3e8aef None admin] PATH_INFO: `/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/3330df65b64c4fffbc3bb687d5c51021/roles/d3cef78b3402418ea77f177cc74fecb1` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.896431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 323/631] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:03:25.897778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff01b8d-22cc-4ca1-91b4-a8cc6c6031e2 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.898134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff01b8d-22cc-4ca1-91b4-a8cc6c6031e2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.898476 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff01b8d-22cc-4ca1-91b4-a8cc6c6031e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.912591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f5bd1-2ee5-463e-b46a-05b02e3e8aef None admin] RBAC: Authorizing `identity:create_grant(project_id=9188023bf51d4737b801e3fbd9af95b1, user_id=3330df65b64c4fffbc3bb687d5c51021, role_id=d3cef78b3402418ea77f177cc74fecb1)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:25.914129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f5bd1-2ee5-463e-b46a-05b02e3e8aef None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:25.917473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-fff01b8d-22cc-4ca1-91b4-a8cc6c6031e2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:25.919912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-fff01b8d-22cc-4ca1-91b4-a8cc6c6031e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:25.935085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 309/632] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:03:25 2022] PUT /identity/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/3330df65b64c4fffbc3bb687d5c51021/roles/d3cef78b3402418ea77f177cc74fecb1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:25.935085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b038468-7f4a-475e-b00d-f08ec6dc2c97 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.935085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b038468-7f4a-475e-b00d-f08ec6dc2c97 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.935085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b038468-7f4a-475e-b00d-f08ec6dc2c97 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.935085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 310/633] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:25.935085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b038468-7f4a-475e-b00d-f08ec6dc2c97 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:25.962232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:26.019958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 324/634] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:26.021830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27826c-6b15-4c86-8176-5f3e05c6c135 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.022062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27826c-6b15-4c86-8176-5f3e05c6c135 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.022338 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27826c-6b15-4c86-8176-5f3e05c6c135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.045249 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-bb27826c-6b15-4c86-8176-5f3e05c6c135 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:26.046620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-bb27826c-6b15-4c86-8176-5f3e05c6c135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:26.102677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b038468-7f4a-475e-b00d-f08ec6dc2c97 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.114266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:26.141836 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 325/635] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:26.142875 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27826c-6b15-4c86-8176-5f3e05c6c135 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.144593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc16e4a-7ead-4ba1-ba31-4b728c5c9b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.145529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc16e4a-7ead-4ba1-ba31-4b728c5c9b9c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.146066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc16e4a-7ead-4ba1-ba31-4b728c5c9b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.146337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc16e4a-7ead-4ba1-ba31-4b728c5c9b9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.159814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc16e4a-7ead-4ba1-ba31-4b728c5c9b9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.161292 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc16e4a-7ead-4ba1-ba31-4b728c5c9b9c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-ecc16e4a-7ead-4ba1-ba31-4b728c5c9b9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:26.184156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:26.185143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:26.186050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:26.186050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:26.186050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:26.188738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 311/636] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:26.194457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc16e4a-7ead-4ba1-ba31-4b728c5c9b9c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.249440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:26.262912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27826c-6b15-4c86-8176-5f3e05c6c135 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.336139 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc16e4a-7ead-4ba1-ba31-4b728c5c9b9c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.348797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284e3fb-b802-45c7-8ae7-f0b950711607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.349447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284e3fb-b802-45c7-8ae7-f0b950711607 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.349742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284e3fb-b802-45c7-8ae7-f0b950711607 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.350233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284e3fb-b802-45c7-8ae7-f0b950711607 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.373780 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284e3fb-b802-45c7-8ae7-f0b950711607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-e284e3fb-b802-45c7-8ae7-f0b950711607 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:26.378929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:26.379926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:26.381099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 326/637] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:26.382507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8aa30-b3fa-4481-bf6f-fbb6b54b0472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.383392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8aa30-b3fa-4481-bf6f-fbb6b54b0472 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.383869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8aa30-b3fa-4481-bf6f-fbb6b54b0472 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.384377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8aa30-b3fa-4481-bf6f-fbb6b54b0472 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.385262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8aa30-b3fa-4481-bf6f-fbb6b54b0472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:26.385788 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8aa30-b3fa-4481-bf6f-fbb6b54b0472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.387400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8aa30-b3fa-4481-bf6f-fbb6b54b0472 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.389107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284e3fb-b802-45c7-8ae7-f0b950711607 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.409277 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 312/638] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:26.416265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8aa30-b3fa-4481-bf6f-fbb6b54b0472 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.446978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:26.541650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284e3fb-b802-45c7-8ae7-f0b950711607 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.546359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8aa30-b3fa-4481-bf6f-fbb6b54b0472 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.583218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a051bb1-19db-4cff-8ffa-d7a82094164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.584129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a051bb1-19db-4cff-8ffa-d7a82094164c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.584274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a051bb1-19db-4cff-8ffa-d7a82094164c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.584480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a051bb1-19db-4cff-8ffa-d7a82094164c None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.585134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a051bb1-19db-4cff-8ffa-d7a82094164c None admin] RBAC: Adding query filter params (name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:26.585437 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a051bb1-19db-4cff-8ffa-d7a82094164c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.586817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a051bb1-19db-4cff-8ffa-d7a82094164c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.589186 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c56e1-7ae8-442d-be29-78a1a7278071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.589900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c56e1-7ae8-442d-be29-78a1a7278071 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.590185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c56e1-7ae8-442d-be29-78a1a7278071 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.590566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c56e1-7ae8-442d-be29-78a1a7278071 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.596902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 327/639] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:26.597967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a051bb1-19db-4cff-8ffa-d7a82094164c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.602810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c56e1-7ae8-442d-be29-78a1a7278071 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.604413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c56e1-7ae8-442d-be29-78a1a7278071 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.612516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-494c56e1-7ae8-442d-be29-78a1a7278071 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:26.616565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:26.617534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:26.618421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:26.618421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:26.618421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:26.618421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:26.618421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:26.618421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:26.618421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:26.618421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:26.618421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 313/640] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:26.618903 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c56e1-7ae8-442d-be29-78a1a7278071 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.712474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:26.737742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c56e1-7ae8-442d-be29-78a1a7278071 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.739511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a051bb1-19db-4cff-8ffa-d7a82094164c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.777029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1aa488-7b98-4065-af8c-cbc0ead83337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.777653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1aa488-7b98-4065-af8c-cbc0ead83337 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.777930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1aa488-7b98-4065-af8c-cbc0ead83337 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.778137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834cf99-e05b-4b12-8bf1-a95078f94d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.778277 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1aa488-7b98-4065-af8c-cbc0ead83337 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.778769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834cf99-e05b-4b12-8bf1-a95078f94d06 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.779038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834cf99-e05b-4b12-8bf1-a95078f94d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.779679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834cf99-e05b-4b12-8bf1-a95078f94d06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.789898 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1aa488-7b98-4065-af8c-cbc0ead83337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:26.791730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834cf99-e05b-4b12-8bf1-a95078f94d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-cf1aa488-7b98-4065-af8c-cbc0ead83337 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:26.795397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:26.797085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:26.798243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:26.798243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:26.798243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:26.798243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:26.798243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:26.798243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:26.798243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:26.798243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:26.798243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:26.798243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:26.798243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:26.798243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-c834cf99-e05b-4b12-8bf1-a95078f94d06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:26.799423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:26.801817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:26.804376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:26.804376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:26.804376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:26.804376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:26.804376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:26.804376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:26.804376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:26.804376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:26.804376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:26.804376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:26.804376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:26.804376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:26.806032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 314/641] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:26.806032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1aa488-7b98-4065-af8c-cbc0ead83337 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.806548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 328/642] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:26.806548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834cf99-e05b-4b12-8bf1-a95078f94d06 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.889051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:26.926150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1aa488-7b98-4065-af8c-cbc0ead83337 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.926353 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834cf99-e05b-4b12-8bf1-a95078f94d06 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:26.965584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e9167-f2ad-4a7b-ae3b-c5f201cb824e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.966192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e9167-f2ad-4a7b-ae3b-c5f201cb824e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.966468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e9167-f2ad-4a7b-ae3b-c5f201cb824e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.966737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e9167-f2ad-4a7b-ae3b-c5f201cb824e None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.967429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e9167-f2ad-4a7b-ae3b-c5f201cb824e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:26.967753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e9167-f2ad-4a7b-ae3b-c5f201cb824e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.969279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e9167-f2ad-4a7b-ae3b-c5f201cb824e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.975855 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c07e6-6f6b-44a8-939e-b5eefe845736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:26.976505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c07e6-6f6b-44a8-939e-b5eefe845736 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:26.976779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c07e6-6f6b-44a8-939e-b5eefe845736 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:26.977050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c07e6-6f6b-44a8-939e-b5eefe845736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:26.982796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 329/643] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:26.989056 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e9167-f2ad-4a7b-ae3b-c5f201cb824e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:26.989570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c07e6-6f6b-44a8-939e-b5eefe845736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:26.991043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c07e6-6f6b-44a8-939e-b5eefe845736 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:26.993929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-8c9c07e6-6f6b-44a8-939e-b5eefe845736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:27.009685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:27.010876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:27.012155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:27.012155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:27.012155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:27.012155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 315/644] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:27.012155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c07e6-6f6b-44a8-939e-b5eefe845736 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:27.127357 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e9167-f2ad-4a7b-ae3b-c5f201cb824e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:27.127592 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c07e6-6f6b-44a8-939e-b5eefe845736 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:27.162726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:27.166478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eec35e-979e-478d-b1e2-1bcc57dc156c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:27.167199 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eec35e-979e-478d-b1e2-1bcc57dc156c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.167652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eec35e-979e-478d-b1e2-1bcc57dc156c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.168131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eec35e-979e-478d-b1e2-1bcc57dc156c None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.168937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eec35e-979e-478d-b1e2-1bcc57dc156c None admin] RBAC: Adding query filter params (name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:27.169241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eec35e-979e-478d-b1e2-1bcc57dc156c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:27.170052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86129037-f4d2-4e84-8776-96f8e64598c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:27.170662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86129037-f4d2-4e84-8776-96f8e64598c6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.171412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eec35e-979e-478d-b1e2-1bcc57dc156c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:27.171514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86129037-f4d2-4e84-8776-96f8e64598c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.172422 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86129037-f4d2-4e84-8776-96f8e64598c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.173109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86129037-f4d2-4e84-8776-96f8e64598c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:27.173418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86129037-f4d2-4e84-8776-96f8e64598c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:27.174781 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86129037-f4d2-4e84-8776-96f8e64598c6 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:27.180624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 316/645] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:27.181313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eec35e-979e-478d-b1e2-1bcc57dc156c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:27.193661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 330/646] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:27.193661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86129037-f4d2-4e84-8776-96f8e64598c6 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:27.250016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:27.306900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eec35e-979e-478d-b1e2-1bcc57dc156c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:27.321560 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86129037-f4d2-4e84-8776-96f8e64598c6 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:27.349983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35736d82-de62-4759-821b-e02878a50219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:27.350595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35736d82-de62-4759-821b-e02878a50219 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.350867 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35736d82-de62-4759-821b-e02878a50219 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.351134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35736d82-de62-4759-821b-e02878a50219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.351858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35736d82-de62-4759-821b-e02878a50219 None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=049399a752ed4646a13f8e4a3a62d8e9) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:27.352201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35736d82-de62-4759-821b-e02878a50219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:27.353625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35736d82-de62-4759-821b-e02878a50219 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:27.360410 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 317/647] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:27 2022] GET /identity/v3/role_assignments?user.id=049399a752ed4646a13f8e4a3a62d8e9&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:27.361232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35736d82-de62-4759-821b-e02878a50219 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:27.362293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d451708-3b83-4110-be43-6b577f8a6066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:27.363057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d451708-3b83-4110-be43-6b577f8a6066 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.363493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d451708-3b83-4110-be43-6b577f8a6066 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.363901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d451708-3b83-4110-be43-6b577f8a6066 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.364587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d451708-3b83-4110-be43-6b577f8a6066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:27.364886 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d451708-3b83-4110-be43-6b577f8a6066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:27.366296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d451708-3b83-4110-be43-6b577f8a6066 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:27.383342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 331/648] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:27.384449 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d451708-3b83-4110-be43-6b577f8a6066 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:27.449242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:27.495415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35736d82-de62-4759-821b-e02878a50219 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:27.515947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d451708-3b83-4110-be43-6b577f8a6066 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:27.526084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:27.548996 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753b0aa-da8b-4d7f-a549-fcaf843b862f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:27.550188 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b0aa-da8b-4d7f-a549-fcaf843b862f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.550527 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b0aa-da8b-4d7f-a549-fcaf843b862f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.551685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b0aa-da8b-4d7f-a549-fcaf843b862f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.564017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753b0aa-da8b-4d7f-a549-fcaf843b862f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:27.566390 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753b0aa-da8b-4d7f-a549-fcaf843b862f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:27.572952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9e346-1cdd-4cbe-b678-31f166047c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:27.573870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9e346-1cdd-4cbe-b678-31f166047c09 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.574440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9e346-1cdd-4cbe-b678-31f166047c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.574885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9e346-1cdd-4cbe-b678-31f166047c09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-8753b0aa-da8b-4d7f-a549-fcaf843b862f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:27.582852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:27.583814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:27.584759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:27.584759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:27.584759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:27.585335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 318/649] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:27.590076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753b0aa-da8b-4d7f-a549-fcaf843b862f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:27.590671 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9e346-1cdd-4cbe-b678-31f166047c09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:27.591562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9e346-1cdd-4cbe-b678-31f166047c09 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-ccb9e346-1cdd-4cbe-b678-31f166047c09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:27.613384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:27.614380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:27.616169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:27.616169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:27.616169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:27.616169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 332/650] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:27.616169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9e346-1cdd-4cbe-b678-31f166047c09 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:27.721046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753b0aa-da8b-4d7f-a549-fcaf843b862f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:27.759214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9e346-1cdd-4cbe-b678-31f166047c09 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:27.761127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da5dda-0994-45a5-b215-ff9ad1e44c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:27.761965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da5dda-0994-45a5-b215-ff9ad1e44c43 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.762251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da5dda-0994-45a5-b215-ff9ad1e44c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.762588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da5dda-0994-45a5-b215-ff9ad1e44c43 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.764291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:27.775071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da5dda-0994-45a5-b215-ff9ad1e44c43 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:27.777041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da5dda-0994-45a5-b215-ff9ad1e44c43 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-34da5dda-0994-45a5-b215-ff9ad1e44c43 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:27.789784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:27.790737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:27.791657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:27.791657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:27.791657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:27.791657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:27.791657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:27.791657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:27.791657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:27.791657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:27.791657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 01:03:27.791657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:27.794399 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 319/651] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:27.795691 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da5dda-0994-45a5-b215-ff9ad1e44c43 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:27.805042 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7bde3-3193-438f-aa3c-20c0a33c220c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:27.805666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7bde3-3193-438f-aa3c-20c0a33c220c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.805948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7bde3-3193-438f-aa3c-20c0a33c220c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.806219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7bde3-3193-438f-aa3c-20c0a33c220c None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.806867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7bde3-3193-438f-aa3c-20c0a33c220c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:27.807173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7bde3-3193-438f-aa3c-20c0a33c220c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:27.808585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7bde3-3193-438f-aa3c-20c0a33c220c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:27.826211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 333/652] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:27.827821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7bde3-3193-438f-aa3c-20c0a33c220c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:27.836626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:27.909226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da5dda-0994-45a5-b215-ff9ad1e44c43 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:27.936422 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7bde3-3193-438f-aa3c-20c0a33c220c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:27.947468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0c1e5-6ba4-43df-a9fc-792b1d76a06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:27.948497 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0c1e5-6ba4-43df-a9fc-792b1d76a06a None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.948977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0c1e5-6ba4-43df-a9fc-792b1d76a06a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.949487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0c1e5-6ba4-43df-a9fc-792b1d76a06a None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.950338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0c1e5-6ba4-43df-a9fc-792b1d76a06a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:27.950934 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0c1e5-6ba4-43df-a9fc-792b1d76a06a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:27.953176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0c1e5-6ba4-43df-a9fc-792b1d76a06a None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:27.966731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 320/653] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:27.969029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0c1e5-6ba4-43df-a9fc-792b1d76a06a None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:27.974275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb939ca-74df-47cd-b754-98d831835f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:27.974897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb939ca-74df-47cd-b754-98d831835f26 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.975176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb939ca-74df-47cd-b754-98d831835f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.975449 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb939ca-74df-47cd-b754-98d831835f26 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.976223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb939ca-74df-47cd-b754-98d831835f26 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:27.976533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb939ca-74df-47cd-b754-98d831835f26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:27.978172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb939ca-74df-47cd-b754-98d831835f26 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:28.001583 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 334/654] 10.222.0.44 () {56 vars in 1139 bytes} [Tue Aug 30 01:03:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:28.002523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb939ca-74df-47cd-b754-98d831835f26 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:28.029423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:28.079628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:28.089382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0c1e5-6ba4-43df-a9fc-792b1d76a06a None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:28.119054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb939ca-74df-47cd-b754-98d831835f26 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:28.129244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574437f9-e86a-41e6-a0bf-d7c71c93e6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:28.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574437f9-e86a-41e6-a0bf-d7c71c93e6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.130437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574437f9-e86a-41e6-a0bf-d7c71c93e6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.130967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574437f9-e86a-41e6-a0bf-d7c71c93e6a6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.138330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574437f9-e86a-41e6-a0bf-d7c71c93e6a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:28.140365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574437f9-e86a-41e6-a0bf-d7c71c93e6a6 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-574437f9-e86a-41e6-a0bf-d7c71c93e6a6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:28.152476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:28.153668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:28.154820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:28.154820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:28.154820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:28.154820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:28.154820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:28.154820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:28.154820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:28.154820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:28.154820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 01:03:28.154820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:28.158666 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 321/655] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:28.159530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574437f9-e86a-41e6-a0bf-d7c71c93e6a6 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:28.162701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177994a-9df4-498f-b670-fd9fc074f46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:28.163349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177994a-9df4-498f-b670-fd9fc074f46f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.163626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177994a-9df4-498f-b670-fd9fc074f46f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.163894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177994a-9df4-498f-b670-fd9fc074f46f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.172482 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177994a-9df4-498f-b670-fd9fc074f46f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:28.173984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177994a-9df4-498f-b670-fd9fc074f46f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-6177994a-9df4-498f-b670-fd9fc074f46f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:28.186372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:28.188589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:28.189521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:28.189521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:28.189521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:28.189521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:28.189521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:28.189521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:28.189521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:28.189521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:28.189521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:03:28.189521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:28.189521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 335/656] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:28.189521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a2430-61f6-401d-a9fe-ffe2b2cb9080 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.190174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a2430-61f6-401d-a9fe-ffe2b2cb9080 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.190174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a2430-61f6-401d-a9fe-ffe2b2cb9080 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.195168 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 336/657] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:28.195168 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a2430-61f6-401d-a9fe-ffe2b2cb9080 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:28.283460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574437f9-e86a-41e6-a0bf-d7c71c93e6a6 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:28.287112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:28.310306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a2430-61f6-401d-a9fe-ffe2b2cb9080 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:28.320081 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd2936-f11c-4078-adef-d5ead234d4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:28.320705 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd2936-f11c-4078-adef-d5ead234d4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.320987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd2936-f11c-4078-adef-d5ead234d4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.321262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd2936-f11c-4078-adef-d5ead234d4a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.345093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:28.350905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799d7df-b63c-4202-8ea6-9afc3f005cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:28.351536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799d7df-b63c-4202-8ea6-9afc3f005cbc None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.351854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799d7df-b63c-4202-8ea6-9afc3f005cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.352243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799d7df-b63c-4202-8ea6-9afc3f005cbc None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.352983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799d7df-b63c-4202-8ea6-9afc3f005cbc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:28.353288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799d7df-b63c-4202-8ea6-9afc3f005cbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:28.354717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799d7df-b63c-4202-8ea6-9afc3f005cbc None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:28.373002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 337/658] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Aug 30 01:03:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:28.373810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799d7df-b63c-4202-8ea6-9afc3f005cbc None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:28.400292 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd2936-f11c-4078-adef-d5ead234d4a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:28.401775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd2936-f11c-4078-adef-d5ead234d4a9 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:28.492291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799d7df-b63c-4202-8ea6-9afc3f005cbc None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:28.512990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 322/659] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:03:28.515073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d53c9-23e4-4f32-88b8-f0535f9f1a9c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.515547 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d53c9-23e4-4f32-88b8-f0535f9f1a9c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.516360 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d53c9-23e4-4f32-88b8-f0535f9f1a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.535490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248de00-1c09-4afe-91b4-762019313c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:28.536067 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248de00-1c09-4afe-91b4-762019313c56 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.536361 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248de00-1c09-4afe-91b4-762019313c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.536637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248de00-1c09-4afe-91b4-762019313c56 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.537373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248de00-1c09-4afe-91b4-762019313c56 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:28.537682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248de00-1c09-4afe-91b4-762019313c56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:28.538537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-0c1d53c9-23e4-4f32-88b8-f0535f9f1a9c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:28.539111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248de00-1c09-4afe-91b4-762019313c56 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:28.540092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0c1d53c9-23e4-4f32-88b8-f0535f9f1a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:28.551332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:28.566057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 338/660] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Aug 30 01:03:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:28.566057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248de00-1c09-4afe-91b4-762019313c56 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:28.594127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:28.616102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 323/661] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:28.617183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d53c9-23e4-4f32-88b8-f0535f9f1a9c None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:28.710043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248de00-1c09-4afe-91b4-762019313c56 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:28.736040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d53c9-23e4-4f32-88b8-f0535f9f1a9c None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:28.749197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ecb94-578b-458c-8f32-de8e7c8c10b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:28.750215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ecb94-578b-458c-8f32-de8e7c8c10b5 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.750863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ecb94-578b-458c-8f32-de8e7c8c10b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.751331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ecb94-578b-458c-8f32-de8e7c8c10b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.772359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858c82a-f7e3-40bc-bcf0-fbe59c164dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:28.773022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858c82a-f7e3-40bc-bcf0-fbe59c164dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.773306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858c82a-f7e3-40bc-bcf0-fbe59c164dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.773579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858c82a-f7e3-40bc-bcf0-fbe59c164dc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.785305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858c82a-f7e3-40bc-bcf0-fbe59c164dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:28.786825 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858c82a-f7e3-40bc-bcf0-fbe59c164dc3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-9858c82a-f7e3-40bc-bcf0-fbe59c164dc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:28.802635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:28.803759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:28.804781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:28.804781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:28.804781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:28.808416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 324/662] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:28.809302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858c82a-f7e3-40bc-bcf0-fbe59c164dc3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:28.834184 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:28.837998 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ecb94-578b-458c-8f32-de8e7c8c10b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:28.839930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ecb94-578b-458c-8f32-de8e7c8c10b5 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:28.860799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:28.959556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 339/663] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:03:28.960570 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ecb94-578b-458c-8f32-de8e7c8c10b5 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:28.966121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858c82a-f7e3-40bc-bcf0-fbe59c164dc3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.005016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77d15f-7876-4d74-bacd-71987eed94af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.005633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77d15f-7876-4d74-bacd-71987eed94af None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.005909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77d15f-7876-4d74-bacd-71987eed94af None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.006212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77d15f-7876-4d74-bacd-71987eed94af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.067387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:29.071774 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ecb94-578b-458c-8f32-de8e7c8c10b5 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.079669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77d15f-7876-4d74-bacd-71987eed94af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:29.081169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77d15f-7876-4d74-bacd-71987eed94af None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:29.110347 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e4631-7378-4b4a-a5eb-35b4fa8911ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.110965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e4631-7378-4b4a-a5eb-35b4fa8911ae None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.111259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e4631-7378-4b4a-a5eb-35b4fa8911ae None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.111512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e4631-7378-4b4a-a5eb-35b4fa8911ae None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.112199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e4631-7378-4b4a-a5eb-35b4fa8911ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:29.112503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e4631-7378-4b4a-a5eb-35b4fa8911ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:29.113894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e4631-7378-4b4a-a5eb-35b4fa8911ae None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:29.115122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:29.126666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 340/664] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:29.127351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e4631-7378-4b4a-a5eb-35b4fa8911ae None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:29.195734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 325/665] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:03:29.197033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77d15f-7876-4d74-bacd-71987eed94af None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:29.249391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e4631-7378-4b4a-a5eb-35b4fa8911ae None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.287600 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4e78d-4b6e-4286-9cc6-f707e1423d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.288385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4e78d-4b6e-4286-9cc6-f707e1423d91 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.288659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4e78d-4b6e-4286-9cc6-f707e1423d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.288927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4e78d-4b6e-4286-9cc6-f707e1423d91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.303057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4e78d-4b6e-4286-9cc6-f707e1423d91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:29.307729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77d15f-7876-4d74-bacd-71987eed94af None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-98d4e78d-4b6e-4286-9cc6-f707e1423d91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:29.308776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:29.309805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:29.310664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 341/666] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:29.311437 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4e78d-4b6e-4286-9cc6-f707e1423d91 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:29.332776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:29.352276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacd8b9-c960-492d-9824-b17532092693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.352910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacd8b9-c960-492d-9824-b17532092693 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.353192 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacd8b9-c960-492d-9824-b17532092693 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.353482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacd8b9-c960-492d-9824-b17532092693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.355443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:29.364909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacd8b9-c960-492d-9824-b17532092693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:29.366380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacd8b9-c960-492d-9824-b17532092693 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-aaacd8b9-c960-492d-9824-b17532092693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:29.383004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:29.384184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:29.385070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:29.385070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:29.385070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:29.385070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 326/667] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:29.385070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacd8b9-c960-492d-9824-b17532092693 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:29.435445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4e78d-4b6e-4286-9cc6-f707e1423d91 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.473819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1253d8-4b77-44d9-9278-02a75cef5977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.474568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1253d8-4b77-44d9-9278-02a75cef5977 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.474845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1253d8-4b77-44d9-9278-02a75cef5977 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.475227 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1253d8-4b77-44d9-9278-02a75cef5977 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.482497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1253d8-4b77-44d9-9278-02a75cef5977 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:29.484124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1253d8-4b77-44d9-9278-02a75cef5977 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:29.493437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacd8b9-c960-492d-9824-b17532092693 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-ff1253d8-4b77-44d9-9278-02a75cef5977 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:29.496222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:29.497267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:29.498210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:29.498210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:29.498210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:29.498210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:29.498210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:29.498210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:29.498210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:29.498210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:29.498210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 342/668] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:29.498210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1253d8-4b77-44d9-9278-02a75cef5977 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:29.533943 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d755b-4d54-495c-a991-60c633ec21c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.534678 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d755b-4d54-495c-a991-60c633ec21c2 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.534942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d755b-4d54-495c-a991-60c633ec21c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.535223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d755b-4d54-495c-a991-60c633ec21c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.546777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d755b-4d54-495c-a991-60c633ec21c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:29.548255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d755b-4d54-495c-a991-60c633ec21c2 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-b64d755b-4d54-495c-a991-60c633ec21c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:29.563550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:29.565195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:29.566670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:29.566670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:29.566670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:29.566670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 327/669] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:29.566670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d755b-4d54-495c-a991-60c633ec21c2 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:29.606225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:29.617201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:29.632941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1253d8-4b77-44d9-9278-02a75cef5977 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.668736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58f807-dea5-4f3b-9840-b6fc4110f355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.669334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58f807-dea5-4f3b-9840-b6fc4110f355 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.669872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58f807-dea5-4f3b-9840-b6fc4110f355 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.670141 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58f807-dea5-4f3b-9840-b6fc4110f355 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.670955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58f807-dea5-4f3b-9840-b6fc4110f355 None admin] RBAC: Adding query filter params (name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:29.671388 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58f807-dea5-4f3b-9840-b6fc4110f355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:29.673000 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58f807-dea5-4f3b-9840-b6fc4110f355 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:29.681622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 343/670] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:29.682919 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78552f1b-7215-4ae2-a3ac-0c8850b1ea8c None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.683050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d755b-4d54-495c-a991-60c633ec21c2 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.683199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78552f1b-7215-4ae2-a3ac-0c8850b1ea8c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.683472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78552f1b-7215-4ae2-a3ac-0c8850b1ea8c None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.687859 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 344/671] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:29.687859 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78552f1b-7215-4ae2-a3ac-0c8850b1ea8c None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:29.725506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bca8fd-f078-4f48-a399-34fc76f477b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.726122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bca8fd-f078-4f48-a399-34fc76f477b4 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.726397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bca8fd-f078-4f48-a399-34fc76f477b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.726668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bca8fd-f078-4f48-a399-34fc76f477b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.727321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bca8fd-f078-4f48-a399-34fc76f477b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:29.727626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bca8fd-f078-4f48-a399-34fc76f477b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:29.729039 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bca8fd-f078-4f48-a399-34fc76f477b4 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:29.741327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 328/672] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:29.742169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bca8fd-f078-4f48-a399-34fc76f477b4 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:29.814637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78552f1b-7215-4ae2-a3ac-0c8850b1ea8c None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.848195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:29.852272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bca8fd-f078-4f48-a399-34fc76f477b4 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:29.854857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fc0a2-5b51-4baf-adf7-41c0b572da52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.855491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fc0a2-5b51-4baf-adf7-41c0b572da52 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.855769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fc0a2-5b51-4baf-adf7-41c0b572da52 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.856079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fc0a2-5b51-4baf-adf7-41c0b572da52 None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.856836 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fc0a2-5b51-4baf-adf7-41c0b572da52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:29.857094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fc0a2-5b51-4baf-adf7-41c0b572da52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:29.858707 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fc0a2-5b51-4baf-adf7-41c0b572da52 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:29.868571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:29.872825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 345/673] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:29.874844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fc0a2-5b51-4baf-adf7-41c0b572da52 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:29.891839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b366b5-041a-43fd-9bb7-9e65b86a5f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:29.892496 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b366b5-041a-43fd-9bb7-9e65b86a5f83 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.892776 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b366b5-041a-43fd-9bb7-9e65b86a5f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.893058 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b366b5-041a-43fd-9bb7-9e65b86a5f83 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.893715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b366b5-041a-43fd-9bb7-9e65b86a5f83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:29.894019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b366b5-041a-43fd-9bb7-9e65b86a5f83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:29.895433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b366b5-041a-43fd-9bb7-9e65b86a5f83 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:29.912466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 329/674] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:29.912466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02180f11-9bf4-4c09-aa35-6b463fc7f3f0 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:29.912466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02180f11-9bf4-4c09-aa35-6b463fc7f3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:29.912466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02180f11-9bf4-4c09-aa35-6b463fc7f3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:29.930190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-02180f11-9bf4-4c09-aa35-6b463fc7f3f0 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:29.931401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-02180f11-9bf4-4c09-aa35-6b463fc7f3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:29.982148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fc0a2-5b51-4baf-adf7-41c0b572da52 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:30.003692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 330/675] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:30.004647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02180f11-9bf4-4c09-aa35-6b463fc7f3f0 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:30.019038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e945db-6b7f-4e5e-a13b-0ecc80460b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:30.019683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e945db-6b7f-4e5e-a13b-0ecc80460b58 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.019965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e945db-6b7f-4e5e-a13b-0ecc80460b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.020266 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e945db-6b7f-4e5e-a13b-0ecc80460b58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.030945 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e945db-6b7f-4e5e-a13b-0ecc80460b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:30.032453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e945db-6b7f-4e5e-a13b-0ecc80460b58 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-97e945db-6b7f-4e5e-a13b-0ecc80460b58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:30.052444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:30.053453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:30.054379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:30.054379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:30.054379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:30.054379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 346/676] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:30.054379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e945db-6b7f-4e5e-a13b-0ecc80460b58 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:30.110458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:30.122075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:30.135876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02180f11-9bf4-4c09-aa35-6b463fc7f3f0 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:30.163942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e945db-6b7f-4e5e-a13b-0ecc80460b58 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:30.173189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcc90e-72ca-4f9a-8c69-eb44a0f2a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:30.174042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcc90e-72ca-4f9a-8c69-eb44a0f2a391 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.174490 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcc90e-72ca-4f9a-8c69-eb44a0f2a391 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.174944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcc90e-72ca-4f9a-8c69-eb44a0f2a391 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.182247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcc90e-72ca-4f9a-8c69-eb44a0f2a391 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:30.183946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcc90e-72ca-4f9a-8c69-eb44a0f2a391 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-4ffcc90e-72ca-4f9a-8c69-eb44a0f2a391 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:30.196725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:30.197822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:30.198953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:30.198953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:30.198953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:30.198953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:30.198953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:30.198953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:30.198953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:30.198953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:30.199474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543aad7-c0de-43f9-bcdb-b3036db9b4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:30.199912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543aad7-c0de-43f9-bcdb-b3036db9b4ad None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.200223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543aad7-c0de-43f9-bcdb-b3036db9b4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.200491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543aad7-c0de-43f9-bcdb-b3036db9b4ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.201208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543aad7-c0de-43f9-bcdb-b3036db9b4ad None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=86cea2f9238b42e0a7cb26bac494a8c7) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:30.201407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 331/677] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:30.201497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543aad7-c0de-43f9-bcdb-b3036db9b4ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:30.202500 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcc90e-72ca-4f9a-8c69-eb44a0f2a391 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:30.203046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543aad7-c0de-43f9-bcdb-b3036db9b4ad None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:30.210979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 347/678] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:30 2022] GET /identity/v3/role_assignments?user.id=86cea2f9238b42e0a7cb26bac494a8c7&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:30.211874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543aad7-c0de-43f9-bcdb-b3036db9b4ad None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:30.337006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcc90e-72ca-4f9a-8c69-eb44a0f2a391 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:30.352754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543aad7-c0de-43f9-bcdb-b3036db9b4ad None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:30.372933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:30.381422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247677d-8609-4106-804c-168b83909bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:30.382071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247677d-8609-4106-804c-168b83909bba None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.382351 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247677d-8609-4106-804c-168b83909bba None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.382616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247677d-8609-4106-804c-168b83909bba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.391466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247677d-8609-4106-804c-168b83909bba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:30.393068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247677d-8609-4106-804c-168b83909bba None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:30.398824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e380af-524f-46c5-b215-0f1bf1dfa341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:30.399447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e380af-524f-46c5-b215-0f1bf1dfa341 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.399732 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e380af-524f-46c5-b215-0f1bf1dfa341 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.400019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e380af-524f-46c5-b215-0f1bf1dfa341 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.400692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e380af-524f-46c5-b215-0f1bf1dfa341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:30.400996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e380af-524f-46c5-b215-0f1bf1dfa341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:30.402372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e380af-524f-46c5-b215-0f1bf1dfa341 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-c247677d-8609-4106-804c-168b83909bba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:30.405627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:30.406629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:30.407516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:30.407516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:30.407516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:30.407516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:30.407516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:30.407516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:30.407516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:30.407516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:30.407516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 332/679] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:30.408645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247677d-8609-4106-804c-168b83909bba None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:30.416152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 348/680] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:30.417518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e380af-524f-46c5-b215-0f1bf1dfa341 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:30.422929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:30.529153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247677d-8609-4106-804c-168b83909bba None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:30.531940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e380af-524f-46c5-b215-0f1bf1dfa341 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:30.568234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb709e0-ac90-4a3e-9b92-95a4f889db19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:30.568851 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb709e0-ac90-4a3e-9b92-95a4f889db19 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.569133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb709e0-ac90-4a3e-9b92-95a4f889db19 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.569409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb709e0-ac90-4a3e-9b92-95a4f889db19 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.570113 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb709e0-ac90-4a3e-9b92-95a4f889db19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:30.570425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb709e0-ac90-4a3e-9b92-95a4f889db19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:30.570809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd999384-f5dd-4ed1-856b-82757810de15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:30.571660 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd999384-f5dd-4ed1-856b-82757810de15 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.571860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb709e0-ac90-4a3e-9b92-95a4f889db19 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:30.572355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd999384-f5dd-4ed1-856b-82757810de15 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.572832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd999384-f5dd-4ed1-856b-82757810de15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.584023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd999384-f5dd-4ed1-856b-82757810de15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:30.585345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 333/681] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:30.586178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb709e0-ac90-4a3e-9b92-95a4f889db19 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-cd999384-f5dd-4ed1-856b-82757810de15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:30.589696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:30.590763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:30.591615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:30.591615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:30.591615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:30.591615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:30.591615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:30.591615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:30.591615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:30.591615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:30.591615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:30.591615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:30.591615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:30.591615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:30.595572 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 349/682] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:30.597685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd999384-f5dd-4ed1-856b-82757810de15 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:30.637405 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:30.663923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:30.714719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb709e0-ac90-4a3e-9b92-95a4f889db19 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:30.728348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd999384-f5dd-4ed1-856b-82757810de15 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:30.754722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4d09b-eedf-40f0-9b46-618b74a03b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:30.755876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4d09b-eedf-40f0-9b46-618b74a03b21 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.756193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4d09b-eedf-40f0-9b46-618b74a03b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.756465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4d09b-eedf-40f0-9b46-618b74a03b21 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.757158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4d09b-eedf-40f0-9b46-618b74a03b21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:30.757463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4d09b-eedf-40f0-9b46-618b74a03b21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:30.758881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4d09b-eedf-40f0-9b46-618b74a03b21 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:30.766634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ecd2f-1c4c-47a4-9bcf-fac4ed8a2bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:30.767280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ecd2f-1c4c-47a4-9bcf-fac4ed8a2bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.767556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ecd2f-1c4c-47a4-9bcf-fac4ed8a2bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.767824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ecd2f-1c4c-47a4-9bcf-fac4ed8a2bb4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.776685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ecd2f-1c4c-47a4-9bcf-fac4ed8a2bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:30.776895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ecd2f-1c4c-47a4-9bcf-fac4ed8a2bb4 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:30.777305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 334/683] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:30.778226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4d09b-eedf-40f0-9b46-618b74a03b21 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-802ecd2f-1c4c-47a4-9bcf-fac4ed8a2bb4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:30.791400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:30.792454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:30.794006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:30.794006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:30.794006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:30.794006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:30.794006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:30.794006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:30.794006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:30.794006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:30.794006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:03:30.794006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:30.794006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 350/684] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:30.794006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ecd2f-1c4c-47a4-9bcf-fac4ed8a2bb4 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:30.897581 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4d09b-eedf-40f0-9b46-618b74a03b21 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:30.898590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:30.905888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ecd2f-1c4c-47a4-9bcf-fac4ed8a2bb4 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:30.929596 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:30.939378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03ac0f-6027-4f2c-8c7e-5d91335d236e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:30.939917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03ac0f-6027-4f2c-8c7e-5d91335d236e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.940226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03ac0f-6027-4f2c-8c7e-5d91335d236e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.940502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03ac0f-6027-4f2c-8c7e-5d91335d236e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.941231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03ac0f-6027-4f2c-8c7e-5d91335d236e None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=92c9d444b3294ed18c3d488cf0a3eeb2) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:30.941548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03ac0f-6027-4f2c-8c7e-5d91335d236e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:30.942970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03ac0f-6027-4f2c-8c7e-5d91335d236e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:30.946857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f270e-414a-472e-9840-422171128d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:30.947461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f270e-414a-472e-9840-422171128d83 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.947733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f270e-414a-472e-9840-422171128d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.948017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f270e-414a-472e-9840-422171128d83 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.948676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f270e-414a-472e-9840-422171128d83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:30.956184 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f270e-414a-472e-9840-422171128d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:30.956389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 335/685] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:30 2022] GET /identity/v3/role_assignments?user.id=92c9d444b3294ed18c3d488cf0a3eeb2&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:30.956389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03ac0f-6027-4f2c-8c7e-5d91335d236e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:30.956568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f270e-414a-472e-9840-422171128d83 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:30.965393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 351/686] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:30.966411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f270e-414a-472e-9840-422171128d83 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:31.088203 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f270e-414a-472e-9840-422171128d83 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:31.114882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03ac0f-6027-4f2c-8c7e-5d91335d236e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:31.139528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddef7c1-5ecc-4e4b-b522-6db2e1c6c406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:31.140386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddef7c1-5ecc-4e4b-b522-6db2e1c6c406 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:31.140736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddef7c1-5ecc-4e4b-b522-6db2e1c6c406 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:31.140843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddef7c1-5ecc-4e4b-b522-6db2e1c6c406 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:31.142749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddef7c1-5ecc-4e4b-b522-6db2e1c6c406 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:31.142749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddef7c1-5ecc-4e4b-b522-6db2e1c6c406 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:31.143213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddef7c1-5ecc-4e4b-b522-6db2e1c6c406 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:31.158605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a87caf-a339-4202-8dc4-f3602628f93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:31.161223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a87caf-a339-4202-8dc4-f3602628f93d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:31.161223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a87caf-a339-4202-8dc4-f3602628f93d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:31.161223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a87caf-a339-4202-8dc4-f3602628f93d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:31.161223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a87caf-a339-4202-8dc4-f3602628f93d None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=534da5b47dda43dabaab2cb6aa683342) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:31.161223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a87caf-a339-4202-8dc4-f3602628f93d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:31.162131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a87caf-a339-4202-8dc4-f3602628f93d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:31.167752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 352/687] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Aug 30 01:03:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:31.168778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 336/688] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:30 2022] GET /identity/v3/role_assignments?user.id=534da5b47dda43dabaab2cb6aa683342&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:31.169623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddef7c1-5ecc-4e4b-b522-6db2e1c6c406 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:31.175070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a87caf-a339-4202-8dc4-f3602628f93d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:31.192979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:31.247157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:31.291814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddef7c1-5ecc-4e4b-b522-6db2e1c6c406 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:31.305679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a87caf-a339-4202-8dc4-f3602628f93d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:31.336313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e198eb-db30-487f-917d-3b71e0ae1278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:31.337449 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e198eb-db30-487f-917d-3b71e0ae1278 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:31.337868 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e198eb-db30-487f-917d-3b71e0ae1278 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:31.338432 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e198eb-db30-487f-917d-3b71e0ae1278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:31.350586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25c56b-7320-4d4c-bf91-c6e9e295808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:31.351329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25c56b-7320-4d4c-bf91-c6e9e295808e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:31.351698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25c56b-7320-4d4c-bf91-c6e9e295808e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:31.352088 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25c56b-7320-4d4c-bf91-c6e9e295808e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:31.353269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e198eb-db30-487f-917d-3b71e0ae1278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:31.354925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e198eb-db30-487f-917d-3b71e0ae1278 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-71e198eb-db30-487f-917d-3b71e0ae1278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:31.375036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:31.376230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:31.377176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:31.377176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:31.377176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:31.379785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 353/689] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:31.386862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e198eb-db30-487f-917d-3b71e0ae1278 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:31.443114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25c56b-7320-4d4c-bf91-c6e9e295808e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:31.444664 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25c56b-7320-4d4c-bf91-c6e9e295808e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:31.484366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:31.514910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:31.525946 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e198eb-db30-487f-917d-3b71e0ae1278 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:31.572184 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fd3c1-add5-4874-97c5-087cae40e0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:31.573044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fd3c1-add5-4874-97c5-087cae40e0eb None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:31.573461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fd3c1-add5-4874-97c5-087cae40e0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:31.573859 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fd3c1-add5-4874-97c5-087cae40e0eb None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:31.574025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 337/690] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 01:03:31.574809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fd3c1-add5-4874-97c5-087cae40e0eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:31.575303 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fd3c1-add5-4874-97c5-087cae40e0eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:31.577472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fd3c1-add5-4874-97c5-087cae40e0eb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:31.580735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25c56b-7320-4d4c-bf91-c6e9e295808e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:31.600908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 354/691] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:31.607143 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fd3c1-add5-4874-97c5-087cae40e0eb None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:31.717247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25c56b-7320-4d4c-bf91-c6e9e295808e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:31.757821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa736f4-42e7-4e0f-971a-1a1db1eea926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:31.758461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa736f4-42e7-4e0f-971a-1a1db1eea926 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:31.758732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa736f4-42e7-4e0f-971a-1a1db1eea926 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:31.759007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa736f4-42e7-4e0f-971a-1a1db1eea926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:31.772179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa736f4-42e7-4e0f-971a-1a1db1eea926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:31.774315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa736f4-42e7-4e0f-971a-1a1db1eea926 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:31.778069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fd3c1-add5-4874-97c5-087cae40e0eb None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-afa736f4-42e7-4e0f-971a-1a1db1eea926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:31.795484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:31.796546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:31.797459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:31.797459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:31.797459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:31.799071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 338/692] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:31.805353 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa736f4-42e7-4e0f-971a-1a1db1eea926 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:31.817630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:31.821650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:31.829178 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677469e-064b-4f5c-b720-1dfc6d575ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:31.830457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677469e-064b-4f5c-b720-1dfc6d575ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:31.830937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677469e-064b-4f5c-b720-1dfc6d575ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:31.831581 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677469e-064b-4f5c-b720-1dfc6d575ef8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:31.852089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677469e-064b-4f5c-b720-1dfc6d575ef8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:31.853558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677469e-064b-4f5c-b720-1dfc6d575ef8 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-b677469e-064b-4f5c-b720-1dfc6d575ef8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:31.880937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:31.881944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:31.882858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:31.882858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:31.882858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:31.882858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:31.882858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:31.882858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:31.882858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:31.882858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:31.882858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:03:31.882858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:31.882858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 355/693] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Aug 30 01:03:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:31.892895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677469e-064b-4f5c-b720-1dfc6d575ef8 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.057178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa736f4-42e7-4e0f-971a-1a1db1eea926 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.125812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677469e-064b-4f5c-b720-1dfc6d575ef8 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.128625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141924d3-43ee-45a5-9973-b6440f08cc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.129236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141924d3-43ee-45a5-9973-b6440f08cc88 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.129515 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141924d3-43ee-45a5-9973-b6440f08cc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.129789 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141924d3-43ee-45a5-9973-b6440f08cc88 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.130439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141924d3-43ee-45a5-9973-b6440f08cc88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:32.130739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141924d3-43ee-45a5-9973-b6440f08cc88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.132328 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141924d3-43ee-45a5-9973-b6440f08cc88 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.160999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 339/694] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:32.170003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141924d3-43ee-45a5-9973-b6440f08cc88 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.202894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5ac30-5d76-4567-b5e4-5e5844010667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.203509 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5ac30-5d76-4567-b5e4-5e5844010667 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.203785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5ac30-5d76-4567-b5e4-5e5844010667 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.204076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5ac30-5d76-4567-b5e4-5e5844010667 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.207530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5ac30-5d76-4567-b5e4-5e5844010667 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:32.207845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5ac30-5d76-4567-b5e4-5e5844010667 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.209478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5ac30-5d76-4567-b5e4-5e5844010667 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.230186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 356/695] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Aug 30 01:03:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:32.235917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5ac30-5d76-4567-b5e4-5e5844010667 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.316443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:32.334389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:32.373533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141924d3-43ee-45a5-9973-b6440f08cc88 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.416497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5ac30-5d76-4567-b5e4-5e5844010667 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.455092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b29ff5-6b04-49b2-85c4-188d011c74e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.455729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b29ff5-6b04-49b2-85c4-188d011c74e4 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.456024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b29ff5-6b04-49b2-85c4-188d011c74e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.456292 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b29ff5-6b04-49b2-85c4-188d011c74e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.464501 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b29ff5-6b04-49b2-85c4-188d011c74e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.465866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b29ff5-6b04-49b2-85c4-188d011c74e4 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.476150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f18869-6dc6-495c-993e-4ce2ace9d275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.477473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f18869-6dc6-495c-993e-4ce2ace9d275 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.478025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f18869-6dc6-495c-993e-4ce2ace9d275 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.478493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f18869-6dc6-495c-993e-4ce2ace9d275 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-09b29ff5-6b04-49b2-85c4-188d011c74e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:32.478849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:32.479734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:32.481083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:32.481083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:32.481083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:32.481083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:32.481083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:32.481083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:32.481083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:32.481083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:32.481083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 340/696] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:32.481833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534fab-2b0b-404f-ad90-78107e5cd0a0 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.481833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534fab-2b0b-404f-ad90-78107e5cd0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.481967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534fab-2b0b-404f-ad90-78107e5cd0a0 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.484029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 341/697] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:32.492024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1534fab-2b0b-404f-ad90-78107e5cd0a0 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.568842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f18869-6dc6-495c-993e-4ce2ace9d275 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.570523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f18869-6dc6-495c-993e-4ce2ace9d275 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.625734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:32.647178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1534fab-2b0b-404f-ad90-78107e5cd0a0 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.683265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:32.694907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec9896-4725-4c01-97de-e95276535f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.695541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec9896-4725-4c01-97de-e95276535f0b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.695925 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec9896-4725-4c01-97de-e95276535f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.696384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec9896-4725-4c01-97de-e95276535f0b None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.697342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec9896-4725-4c01-97de-e95276535f0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:32.697876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec9896-4725-4c01-97de-e95276535f0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.699344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec9896-4725-4c01-97de-e95276535f0b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.702696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 357/698] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:03:32.704604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f19e01-fed1-4a28-963c-681c8d6e9137 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.705034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f19e01-fed1-4a28-963c-681c8d6e9137 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.705500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f19e01-fed1-4a28-963c-681c8d6e9137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.714592 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 342/699] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:32.715480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec9896-4725-4c01-97de-e95276535f0b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.732939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-13f19e01-fed1-4a28-963c-681c8d6e9137 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:32.734984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-13f19e01-fed1-4a28-963c-681c8d6e9137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:32.816155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 358/700] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:32.817670 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f19e01-fed1-4a28-963c-681c8d6e9137 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:32.879953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec9896-4725-4c01-97de-e95276535f0b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:32.943452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fe584-ab3a-4888-b505-ce24f301c7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.944455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fe584-ab3a-4888-b505-ce24f301c7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.945195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fe584-ab3a-4888-b505-ce24f301c7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.945435 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fe584-ab3a-4888-b505-ce24f301c7d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.971011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fe584-ab3a-4888-b505-ce24f301c7d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.973663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fe584-ab3a-4888-b505-ce24f301c7d7 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-482fe584-ab3a-4888-b505-ce24f301c7d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:33.011772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:33.012829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:33.015464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:33.015464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:33.015464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:33.015464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 343/701] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:33.019017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:33.019911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fe584-ab3a-4888-b505-ce24f301c7d7 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.075052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f19e01-fed1-4a28-963c-681c8d6e9137 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.114381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:33.172850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432e950-cb95-4eff-9b50-76f7464d4090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.173516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432e950-cb95-4eff-9b50-76f7464d4090 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.173690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432e950-cb95-4eff-9b50-76f7464d4090 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.173940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432e950-cb95-4eff-9b50-76f7464d4090 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.174689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432e950-cb95-4eff-9b50-76f7464d4090 None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=3330df65b64c4fffbc3bb687d5c51021) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:33.175002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432e950-cb95-4eff-9b50-76f7464d4090 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.176495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432e950-cb95-4eff-9b50-76f7464d4090 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.185662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 359/702] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:32 2022] GET /identity/v3/role_assignments?user.id=3330df65b64c4fffbc3bb687d5c51021&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 608 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:33.186904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432e950-cb95-4eff-9b50-76f7464d4090 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.217098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fe584-ab3a-4888-b505-ce24f301c7d7 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.269122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1769b5-97b7-439d-8b12-3c1d21f16a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.269690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1769b5-97b7-439d-8b12-3c1d21f16a66 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.270061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1769b5-97b7-439d-8b12-3c1d21f16a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.270341 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1769b5-97b7-439d-8b12-3c1d21f16a66 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.271095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1769b5-97b7-439d-8b12-3c1d21f16a66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:33.271408 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1769b5-97b7-439d-8b12-3c1d21f16a66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.273182 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1769b5-97b7-439d-8b12-3c1d21f16a66 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.289527 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 344/703] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:33.295780 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1769b5-97b7-439d-8b12-3c1d21f16a66 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.335279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432e950-cb95-4eff-9b50-76f7464d4090 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.383024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d188cc7-16b9-4b39-9150-c8010a5ebfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.384619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d188cc7-16b9-4b39-9150-c8010a5ebfc9 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.385138 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d188cc7-16b9-4b39-9150-c8010a5ebfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.386365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d188cc7-16b9-4b39-9150-c8010a5ebfc9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.387593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d188cc7-16b9-4b39-9150-c8010a5ebfc9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.389683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d188cc7-16b9-4b39-9150-c8010a5ebfc9 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-7d188cc7-16b9-4b39-9150-c8010a5ebfc9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:33.407153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:33.408277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:03:33.409126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:33.416692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 360/704] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:33 2022] GET /identity/v3/services/network => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:33.421346 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d188cc7-16b9-4b39-9150-c8010a5ebfc9 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.427737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:33.429455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:33.433711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1769b5-97b7-439d-8b12-3c1d21f16a66 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.493016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6d3be-2b60-4a8b-a41d-f77984c0ed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.493672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6d3be-2b60-4a8b-a41d-f77984c0ed9e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.493947 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6d3be-2b60-4a8b-a41d-f77984c0ed9e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.494220 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6d3be-2b60-4a8b-a41d-f77984c0ed9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.502802 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6d3be-2b60-4a8b-a41d-f77984c0ed9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.504378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6d3be-2b60-4a8b-a41d-f77984c0ed9e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-1bb6d3be-2b60-4a8b-a41d-f77984c0ed9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:33.521429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:33.522475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:33.524312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:33.524312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:33.524312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:33.524312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:33.524312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:33.524312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:33.524312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:33.524312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:33.524312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 345/705] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:33.524312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f088c82-5c7d-4069-b39e-a6da3a525aa3 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.524865 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f088c82-5c7d-4069-b39e-a6da3a525aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.524865 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f088c82-5c7d-4069-b39e-a6da3a525aa3 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.526908 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 346/706] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:33.529708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f088c82-5c7d-4069-b39e-a6da3a525aa3 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.572404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d188cc7-16b9-4b39-9150-c8010a5ebfc9 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.616904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f913e-ca5d-4e33-a45a-dfa25bd66205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.616904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f913e-ca5d-4e33-a45a-dfa25bd66205 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.617201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f913e-ca5d-4e33-a45a-dfa25bd66205 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.617201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f913e-ca5d-4e33-a45a-dfa25bd66205 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.618012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f913e-ca5d-4e33-a45a-dfa25bd66205 None admin] RBAC: Adding query filter params (name=network) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:33.618321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f913e-ca5d-4e33-a45a-dfa25bd66205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.619730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f913e-ca5d-4e33-a45a-dfa25bd66205 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.626026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 361/707] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:03:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:33.626821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797d6b5-2476-45d5-b694-992902b673c1 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.626821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797d6b5-2476-45d5-b694-992902b673c1 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.627094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797d6b5-2476-45d5-b694-992902b673c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.652839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7797d6b5-2476-45d5-b694-992902b673c1 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:33.653780 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7797d6b5-2476-45d5-b694-992902b673c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:33.660907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f088c82-5c7d-4069-b39e-a6da3a525aa3 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.709217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6abf4-9dcf-4219-b7a3-d41f04eb17a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.709217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6abf4-9dcf-4219-b7a3-d41f04eb17a3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.709217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6abf4-9dcf-4219-b7a3-d41f04eb17a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.709217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6abf4-9dcf-4219-b7a3-d41f04eb17a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.709569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6abf4-9dcf-4219-b7a3-d41f04eb17a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:33.709569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6abf4-9dcf-4219-b7a3-d41f04eb17a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.710503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6abf4-9dcf-4219-b7a3-d41f04eb17a3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.724717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 347/708] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:33.725703 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6abf4-9dcf-4219-b7a3-d41f04eb17a3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.742410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:33.749700 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 362/709] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:33.750019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:33.756653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797d6b5-2476-45d5-b694-992902b673c1 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.844919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6abf4-9dcf-4219-b7a3-d41f04eb17a3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.886635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797d6b5-2476-45d5-b694-992902b673c1 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.893573 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9cfaf-ce9e-44a8-98eb-35880b50f3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.894252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9cfaf-ce9e-44a8-98eb-35880b50f3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.894759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9cfaf-ce9e-44a8-98eb-35880b50f3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.895103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9cfaf-ce9e-44a8-98eb-35880b50f3f4 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.896391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9cfaf-ce9e-44a8-98eb-35880b50f3f4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:33.896754 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9cfaf-ce9e-44a8-98eb-35880b50f3f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.898842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9cfaf-ce9e-44a8-98eb-35880b50f3f4 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.908327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 348/710] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:03:33 2022] GET /identity/v3/services?type=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:33.909637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa1cff-e989-4617-831f-5d43a0b6ed66 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.909919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa1cff-e989-4617-831f-5d43a0b6ed66 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.910186 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa1cff-e989-4617-831f-5d43a0b6ed66 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.911709 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 349/711] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:33.917157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527c7ca-df5c-4c6a-8b6a-9e6c85f3738a None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.917452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527c7ca-df5c-4c6a-8b6a-9e6c85f3738a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.917701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527c7ca-df5c-4c6a-8b6a-9e6c85f3738a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.938996 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-a527c7ca-df5c-4c6a-8b6a-9e6c85f3738a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:33.940396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a527c7ca-df5c-4c6a-8b6a-9e6c85f3738a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:33.952591 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529779ee-bc5f-415b-9f24-92c3fabf658f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.953267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529779ee-bc5f-415b-9f24-92c3fabf658f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.953537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529779ee-bc5f-415b-9f24-92c3fabf658f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.953797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529779ee-bc5f-415b-9f24-92c3fabf658f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.954580 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529779ee-bc5f-415b-9f24-92c3fabf658f None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=049399a752ed4646a13f8e4a3a62d8e9) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:33.954883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529779ee-bc5f-415b-9f24-92c3fabf658f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:33.956410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529779ee-bc5f-415b-9f24-92c3fabf658f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:33.970607 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 363/712] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:33 2022] GET /identity/v3/role_assignments?user.id=049399a752ed4646a13f8e4a3a62d8e9&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:34.035510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 350/713] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:03:34.086837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529779ee-bc5f-415b-9f24-92c3fabf658f None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:34.159483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:34.203913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529779ee-bc5f-415b-9f24-92c3fabf658f None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:34.240271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435fc2f-be07-41a4-ad6f-083ae3b9f560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:34.240827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435fc2f-be07-41a4-ad6f-083ae3b9f560 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:34.241100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435fc2f-be07-41a4-ad6f-083ae3b9f560 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:34.241368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435fc2f-be07-41a4-ad6f-083ae3b9f560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:34.253830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435fc2f-be07-41a4-ad6f-083ae3b9f560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:34.255356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435fc2f-be07-41a4-ad6f-083ae3b9f560 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-f435fc2f-be07-41a4-ad6f-083ae3b9f560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:34.271061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:34.273620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:34.275743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:34.275743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:34.275743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:34.275743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 364/714] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:34.281392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527c7ca-df5c-4c6a-8b6a-9e6c85f3738a None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:34.298452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:34.339302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435fc2f-be07-41a4-ad6f-083ae3b9f560 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:34.450237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527c7ca-df5c-4c6a-8b6a-9e6c85f3738a None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:34.490619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435fc2f-be07-41a4-ad6f-083ae3b9f560 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:34.513576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:34.520721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd70bac-076d-42e9-9f78-6c7e64650b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:34.521354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd70bac-076d-42e9-9f78-6c7e64650b88 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:34.521634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd70bac-076d-42e9-9f78-6c7e64650b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:34.522423 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd70bac-076d-42e9-9f78-6c7e64650b88 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:34.522595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd70bac-076d-42e9-9f78-6c7e64650b88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:34.522859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd70bac-076d-42e9-9f78-6c7e64650b88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:34.524284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd70bac-076d-42e9-9f78-6c7e64650b88 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:34.529497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee36f5-58f7-4edd-87e4-b03e05ed7114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:34.530709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee36f5-58f7-4edd-87e4-b03e05ed7114 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:34.531020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee36f5-58f7-4edd-87e4-b03e05ed7114 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:34.531289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee36f5-58f7-4edd-87e4-b03e05ed7114 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:34.532333 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee36f5-58f7-4edd-87e4-b03e05ed7114 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:34.533314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee36f5-58f7-4edd-87e4-b03e05ed7114 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-69ee36f5-58f7-4edd-87e4-b03e05ed7114 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:34.542824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:34.543844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:34.545242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 365/715] 10.222.0.44 () {56 vars in 1129 bytes} [Tue Aug 30 01:03:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:34.546402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 351/716] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:34.553277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee36f5-58f7-4edd-87e4-b03e05ed7114 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:34.553530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd70bac-076d-42e9-9f78-6c7e64650b88 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:34.661904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee36f5-58f7-4edd-87e4-b03e05ed7114 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:34.669054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:34.699397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd70bac-076d-42e9-9f78-6c7e64650b88 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:34.730249 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e561a9-4664-4048-89f1-919bc84d07aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:34.730868 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e561a9-4664-4048-89f1-919bc84d07aa None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:34.731143 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e561a9-4664-4048-89f1-919bc84d07aa None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:34.731407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e561a9-4664-4048-89f1-919bc84d07aa None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:34.732089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e561a9-4664-4048-89f1-919bc84d07aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:34.733566 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e561a9-4664-4048-89f1-919bc84d07aa None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:34.772793 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e13a67-772e-4a0d-9c94-575871db6302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:34.773323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e13a67-772e-4a0d-9c94-575871db6302 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:34.773596 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e13a67-772e-4a0d-9c94-575871db6302 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:34.773859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e13a67-772e-4a0d-9c94-575871db6302 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:34.775371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 366/717] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:03:34 2022] POST /identity/v3/users => generated 254 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:34.775818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e13a67-772e-4a0d-9c94-575871db6302 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:34.776151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e13a67-772e-4a0d-9c94-575871db6302 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:34.777594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e13a67-772e-4a0d-9c94-575871db6302 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:34.784014 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 352/718] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Aug 30 01:03:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:34.789737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e561a9-4664-4048-89f1-919bc84d07aa None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:34.852427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:34.922814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e561a9-4664-4048-89f1-919bc84d07aa None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:34.961997 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca67a8-084c-4df1-9ad8-94d27025148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:34.962625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca67a8-084c-4df1-9ad8-94d27025148e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:34.962904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca67a8-084c-4df1-9ad8-94d27025148e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:34.963176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca67a8-084c-4df1-9ad8-94d27025148e None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:34.963837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca67a8-084c-4df1-9ad8-94d27025148e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:34.964176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca67a8-084c-4df1-9ad8-94d27025148e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:34.965619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca67a8-084c-4df1-9ad8-94d27025148e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:34.972706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 367/719] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Aug 30 01:03:34 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:35.769583 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb53d8-bf79-41df-8da5-d460707aa066 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:35.770115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb53d8-bf79-41df-8da5-d460707aa066 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:35.770115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb53d8-bf79-41df-8da5-d460707aa066 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:35.771803 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 353/720] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:35.777314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2451a-950a-4bcb-9a84-c713066abed0 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:35.777810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2451a-950a-4bcb-9a84-c713066abed0 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:35.777888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2451a-950a-4bcb-9a84-c713066abed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:35.809191 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-0cf2451a-950a-4bcb-9a84-c713066abed0 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:35.810426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0cf2451a-950a-4bcb-9a84-c713066abed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:35.911343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 368/721] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:36.325311 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb53d8-bf79-41df-8da5-d460707aa066 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.411089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895d11b-fac4-45f2-a841-4396334dc583 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.411480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895d11b-fac4-45f2-a841-4396334dc583 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.411657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895d11b-fac4-45f2-a841-4396334dc583 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.413307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 369/722] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:36.419043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69fe9d-ad0e-4335-afee-597820371fef None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.419572 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69fe9d-ad0e-4335-afee-597820371fef None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.419848 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69fe9d-ad0e-4335-afee-597820371fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.442387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-5a69fe9d-ad0e-4335-afee-597820371fef None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:36.443587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5a69fe9d-ad0e-4335-afee-597820371fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:36.494438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb53d8-bf79-41df-8da5-d460707aa066 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.495158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:36.538408 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07c38f-cc13-4c86-8c3b-1f5d2967c90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.539073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07c38f-cc13-4c86-8c3b-1f5d2967c90e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.539494 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07c38f-cc13-4c86-8c3b-1f5d2967c90e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.539764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07c38f-cc13-4c86-8c3b-1f5d2967c90e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.548607 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:36.557342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07c38f-cc13-4c86-8c3b-1f5d2967c90e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-5b07c38f-cc13-4c86-8c3b-1f5d2967c90e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:36.563169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:36.564206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:36.565019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:36.565019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:36.565019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:36.565019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:36.565019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:36.565019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:36.565019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:36.565019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:36.565019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:36.565019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:36.565019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:36.565019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:36.568350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 354/723] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:36.570600 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07c38f-cc13-4c86-8c3b-1f5d2967c90e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.606681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 370/724] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:36.610682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b0451-1ce0-419f-bb75-947e493f80e2 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.611360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b0451-1ce0-419f-bb75-947e493f80e2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.611844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b0451-1ce0-419f-bb75-947e493f80e2 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.613659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 371/725] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:36.627681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da08a8b-603e-4a5c-9288-696683791d98 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.628340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da08a8b-603e-4a5c-9288-696683791d98 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.628809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da08a8b-603e-4a5c-9288-696683791d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.667036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-8da08a8b-603e-4a5c-9288-696683791d98 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:36.669637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8da08a8b-603e-4a5c-9288-696683791d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:36.761053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07c38f-cc13-4c86-8c3b-1f5d2967c90e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:36.804282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca8f26-6526-41b3-8627-7f766deb11d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:36.805910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca8f26-6526-41b3-8627-7f766deb11d7 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:36.806333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca8f26-6526-41b3-8627-7f766deb11d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:36.806681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 372/726] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:36.812548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca8f26-6526-41b3-8627-7f766deb11d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:36.813471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca8f26-6526-41b3-8627-7f766deb11d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:36.813927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca8f26-6526-41b3-8627-7f766deb11d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:36.815486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca8f26-6526-41b3-8627-7f766deb11d7 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:36.831670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 355/727] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:36.837134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da08a8b-603e-4a5c-9288-696683791d98 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.923344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca8f26-6526-41b3-8627-7f766deb11d7 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:36.981967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:36.998462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da08a8b-603e-4a5c-9288-696683791d98 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:37.003431 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:37.070222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde8b62-54ed-46e0-9a61-2f7d8e91e2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:37.070872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde8b62-54ed-46e0-9a61-2f7d8e91e2cc None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.071169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde8b62-54ed-46e0-9a61-2f7d8e91e2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.071440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde8b62-54ed-46e0-9a61-2f7d8e91e2cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.091932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca8f26-6526-41b3-8627-7f766deb11d7 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:37.093303 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde8b62-54ed-46e0-9a61-2f7d8e91e2cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:37.094453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde8b62-54ed-46e0-9a61-2f7d8e91e2cc None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-9dde8b62-54ed-46e0-9a61-2f7d8e91e2cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:37.121754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:37.122769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:37.124235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:37.124235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:37.124235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:37.129893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 373/728] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:37.135797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde8b62-54ed-46e0-9a61-2f7d8e91e2cc None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:37.155191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045f848-8023-479e-9b19-0c2caac8784b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:37.156213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045f848-8023-479e-9b19-0c2caac8784b None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.156725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045f848-8023-479e-9b19-0c2caac8784b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.157279 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045f848-8023-479e-9b19-0c2caac8784b None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.158155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045f848-8023-479e-9b19-0c2caac8784b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:37.159815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045f848-8023-479e-9b19-0c2caac8784b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:37.187400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 356/729] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:03:36 2022] POST /identity/v3/services => generated 240 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:03:37.215084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045f848-8023-479e-9b19-0c2caac8784b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:37.331741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde8b62-54ed-46e0-9a61-2f7d8e91e2cc None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:37.387183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6b2cd-813a-441f-8881-f676c3f63a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:37.387823 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6b2cd-813a-441f-8881-f676c3f63a45 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.388146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6b2cd-813a-441f-8881-f676c3f63a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.388417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6b2cd-813a-441f-8881-f676c3f63a45 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.389642 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6b2cd-813a-441f-8881-f676c3f63a45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:37.390366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6b2cd-813a-441f-8881-f676c3f63a45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:37.391360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6b2cd-813a-441f-8881-f676c3f63a45 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:37.394275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:37.408740 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 374/730] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:37.410372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a16763-dea3-403c-92f1-a0c57d60e06b None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.410839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a16763-dea3-403c-92f1-a0c57d60e06b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.411292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a16763-dea3-403c-92f1-a0c57d60e06b None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.411462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045f848-8023-479e-9b19-0c2caac8784b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:37.413288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 375/731] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:37.416501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a16763-dea3-403c-92f1-a0c57d60e06b None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:37.443576 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:37.484942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f2346-b9dc-49e0-9bca-b986a2d2801b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:37.485970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f2346-b9dc-49e0-9bca-b986a2d2801b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.486380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f2346-b9dc-49e0-9bca-b986a2d2801b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.486772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f2346-b9dc-49e0-9bca-b986a2d2801b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.513732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f2346-b9dc-49e0-9bca-b986a2d2801b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-5c9f2346-b9dc-49e0-9bca-b986a2d2801b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:37.520234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:37.527140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 357/732] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:37.527980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c99a66-050b-4ed7-9f69-f8ab523b4f75 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.528765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c99a66-050b-4ed7-9f69-f8ab523b4f75 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.528765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c99a66-050b-4ed7-9f69-f8ab523b4f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.559912 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-55c99a66-050b-4ed7-9f69-f8ab523b4f75 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:37.562738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-55c99a66-050b-4ed7-9f69-f8ab523b4f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:37.599696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a16763-dea3-403c-92f1-a0c57d60e06b None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:37.645231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9ebff-b4e4-42b4-a696-6368999fe3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:37.646223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ebff-b4e4-42b4-a696-6368999fe3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.646711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ebff-b4e4-42b4-a696-6368999fe3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.647165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ebff-b4e4-42b4-a696-6368999fe3b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.661189 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9ebff-b4e4-42b4-a696-6368999fe3b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:37.663931 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9ebff-b4e4-42b4-a696-6368999fe3b0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-88d9ebff-b4e4-42b4-a696-6368999fe3b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:37.677753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:37.678867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:37.679883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:37.679883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:37.679883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:37.679883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:37.679883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:37.679883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:37.679883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:37.679883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:37.679883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:03:37.679883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:37.682484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 376/734] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:37.682677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 358/734] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:37 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:37.686864 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c99a66-050b-4ed7-9f69-f8ab523b4f75 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:37.689164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9ebff-b4e4-42b4-a696-6368999fe3b0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:37.775379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:37.837793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9ebff-b4e4-42b4-a696-6368999fe3b0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:37.847710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c99a66-050b-4ed7-9f69-f8ab523b4f75 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:37.865126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:37.877295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fabd2-85a8-44d1-8a1c-3880980dd1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:37.877917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fabd2-85a8-44d1-8a1c-3880980dd1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.878198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fabd2-85a8-44d1-8a1c-3880980dd1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.878472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fabd2-85a8-44d1-8a1c-3880980dd1d5 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.879216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fabd2-85a8-44d1-8a1c-3880980dd1d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:37.879524 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fabd2-85a8-44d1-8a1c-3880980dd1d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:37.881011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fabd2-85a8-44d1-8a1c-3880980dd1d5 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:37.896755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4d6fd-1967-44db-9d12-3e4cc4f76495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:37.897715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4d6fd-1967-44db-9d12-3e4cc4f76495 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.898162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4d6fd-1967-44db-9d12-3e4cc4f76495 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.898574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4d6fd-1967-44db-9d12-3e4cc4f76495 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.899432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4d6fd-1967-44db-9d12-3e4cc4f76495 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:37.899874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4d6fd-1967-44db-9d12-3e4cc4f76495 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:37.901402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 377/735] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Aug 30 01:03:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:37.901659 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4d6fd-1967-44db-9d12-3e4cc4f76495 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:37.902689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456616dc-6253-4cf3-92f2-db71a8b3f338 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.902970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456616dc-6253-4cf3-92f2-db71a8b3f338 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.903241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456616dc-6253-4cf3-92f2-db71a8b3f338 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.904741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 378/736] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:37.910796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4ec30-afaa-4b82-9c76-3978b9c7b5ff None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.911159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4ec30-afaa-4b82-9c76-3978b9c7b5ff None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.911510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4ec30-afaa-4b82-9c76-3978b9c7b5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.913400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 359/737] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:37.914427 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4d6fd-1967-44db-9d12-3e4cc4f76495 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:37.932225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-44e4ec30-afaa-4b82-9c76-3978b9c7b5ff None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:37.933605 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-44e4ec30-afaa-4b82-9c76-3978b9c7b5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:38.012394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 379/738] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:37 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:03:38.013271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4ec30-afaa-4b82-9c76-3978b9c7b5ff None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:38.034048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4d6fd-1967-44db-9d12-3e4cc4f76495 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:38.087298 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:38.095601 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a281e7a-d62f-4aba-8b84-79547b7ba26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:38.097159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a281e7a-d62f-4aba-8b84-79547b7ba26d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:38.097893 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a281e7a-d62f-4aba-8b84-79547b7ba26d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:38.098605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a281e7a-d62f-4aba-8b84-79547b7ba26d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:38.147775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4ec30-afaa-4b82-9c76-3978b9c7b5ff None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:38.183422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:38.185554 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d94c97-0170-44d4-a65f-26a8b7e930d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:38.186693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d94c97-0170-44d4-a65f-26a8b7e930d3 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:38.187021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d94c97-0170-44d4-a65f-26a8b7e930d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:38.187326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d94c97-0170-44d4-a65f-26a8b7e930d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:38.198446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d94c97-0170-44d4-a65f-26a8b7e930d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:38.200048 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d94c97-0170-44d4-a65f-26a8b7e930d3 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-39d94c97-0170-44d4-a65f-26a8b7e930d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:38.216834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:38.219173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:38.221698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:38.221698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:38.221698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:38.221698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 380/739] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:38.221698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d94c97-0170-44d4-a65f-26a8b7e930d3 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:38.245998 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a281e7a-d62f-4aba-8b84-79547b7ba26d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:38.249470 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a281e7a-d62f-4aba-8b84-79547b7ba26d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:38.387905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 360/740] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:37 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:03:38.389710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a281e7a-d62f-4aba-8b84-79547b7ba26d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:38.413858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:38.427228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d94c97-0170-44d4-a65f-26a8b7e930d3 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:38.470999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:38.486798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49218f-1e00-480b-b8f0-5805ac2f3a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:38.487429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49218f-1e00-480b-b8f0-5805ac2f3a3a None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:38.487705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49218f-1e00-480b-b8f0-5805ac2f3a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:38.487974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49218f-1e00-480b-b8f0-5805ac2f3a3a None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:38.488669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49218f-1e00-480b-b8f0-5805ac2f3a3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:38.490041 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49218f-1e00-480b-b8f0-5805ac2f3a3a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:38.508178 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 381/741] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:03:38 2022] POST /identity/v3/services => generated 251 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:38.509057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49218f-1e00-480b-b8f0-5805ac2f3a3a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:38.537302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a281e7a-d62f-4aba-8b84-79547b7ba26d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:38.646363 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ba250-bc99-4e1c-8866-ff7b62e15a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:38.647345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ba250-bc99-4e1c-8866-ff7b62e15a52 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:38.647542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ba250-bc99-4e1c-8866-ff7b62e15a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:38.647929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ba250-bc99-4e1c-8866-ff7b62e15a52 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:38.648777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ba250-bc99-4e1c-8866-ff7b62e15a52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:38.649185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ba250-bc99-4e1c-8866-ff7b62e15a52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:38.650737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ba250-bc99-4e1c-8866-ff7b62e15a52 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:38.672168 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 361/742] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:38.673365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ba250-bc99-4e1c-8866-ff7b62e15a52 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:38.702186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49218f-1e00-480b-b8f0-5805ac2f3a3a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:38.746679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b0563-a63f-4354-9fdc-6e830dbe9ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:38.747435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b0563-a63f-4354-9fdc-6e830dbe9ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:38.747752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b0563-a63f-4354-9fdc-6e830dbe9ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:38.756052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b0563-a63f-4354-9fdc-6e830dbe9ac1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:38.770877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b0563-a63f-4354-9fdc-6e830dbe9ac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:38.773814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b0563-a63f-4354-9fdc-6e830dbe9ac1 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-463b0563-a63f-4354-9fdc-6e830dbe9ac1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:38.797375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:38.798419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:38.799484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:38.799484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:38.799484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:38.799484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 382/743] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:38.799718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b0563-a63f-4354-9fdc-6e830dbe9ac1 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:38.812959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:38.860907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ba250-bc99-4e1c-8866-ff7b62e15a52 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:38.916569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:38.933759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17d41a-4408-4859-8cf6-cddf13c5b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:38.934764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17d41a-4408-4859-8cf6-cddf13c5b29e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:38.935234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17d41a-4408-4859-8cf6-cddf13c5b29e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:38.935797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17d41a-4408-4859-8cf6-cddf13c5b29e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:38.955337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17d41a-4408-4859-8cf6-cddf13c5b29e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:38.957730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b0563-a63f-4354-9fdc-6e830dbe9ac1 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-9c17d41a-4408-4859-8cf6-cddf13c5b29e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:38.964882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:38.966690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:38.968374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:38.968374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:38.968374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:38.968374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:38.968374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:38.968374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:38.968374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:38.968374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:38.968374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:38.968374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:38.968374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:38.968374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:38.972105 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 362/744] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:38.973586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17d41a-4408-4859-8cf6-cddf13c5b29e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.033336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8725db-6948-4d86-9440-03e20907284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.035336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8725db-6948-4d86-9440-03e20907284b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.036367 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8725db-6948-4d86-9440-03e20907284b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.037059 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8725db-6948-4d86-9440-03e20907284b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.054443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8725db-6948-4d86-9440-03e20907284b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.058395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8725db-6948-4d86-9440-03e20907284b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-3d8725db-6948-4d86-9440-03e20907284b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:39.085185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:39.086370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:39.087513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:39.087513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:39.087513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:39.087513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:39.087513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:39.087513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:39.087513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:39.087513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:39.087513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:03:39.087513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:39.091835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 383/745] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Aug 30 01:03:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:39.093646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8725db-6948-4d86-9440-03e20907284b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.186339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17d41a-4408-4859-8cf6-cddf13c5b29e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.188481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:39.240865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8725db-6948-4d86-9440-03e20907284b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.253053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d15a14-c29f-4029-979b-67c6161b55b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.254110 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d15a14-c29f-4029-979b-67c6161b55b6 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.254612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d15a14-c29f-4029-979b-67c6161b55b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.255100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d15a14-c29f-4029-979b-67c6161b55b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.256293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d15a14-c29f-4029-979b-67c6161b55b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:39.256829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d15a14-c29f-4029-979b-67c6161b55b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.259133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d15a14-c29f-4029-979b-67c6161b55b6 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.287312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 363/746] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:39.287312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d15a14-c29f-4029-979b-67c6161b55b6 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.290296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af159b-2726-43ce-9af9-b3dfc6bccbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.291349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af159b-2726-43ce-9af9-b3dfc6bccbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.291857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af159b-2726-43ce-9af9-b3dfc6bccbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.292406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af159b-2726-43ce-9af9-b3dfc6bccbf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.293288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af159b-2726-43ce-9af9-b3dfc6bccbf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:39.293798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af159b-2726-43ce-9af9-b3dfc6bccbf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.296807 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af159b-2726-43ce-9af9-b3dfc6bccbf1 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.311738 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 384/747] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:39.313186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af159b-2726-43ce-9af9-b3dfc6bccbf1 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.381596 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:39.444607 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af159b-2726-43ce-9af9-b3dfc6bccbf1 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.478226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:39.479199 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d15a14-c29f-4029-979b-67c6161b55b6 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.487441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc1bcb-d1b4-4ad0-a997-dbda8e26a4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.488503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc1bcb-d1b4-4ad0-a997-dbda8e26a4da None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.488989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc1bcb-d1b4-4ad0-a997-dbda8e26a4da None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.489461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc1bcb-d1b4-4ad0-a997-dbda8e26a4da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.497292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc1bcb-d1b4-4ad0-a997-dbda8e26a4da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.499386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc1bcb-d1b4-4ad0-a997-dbda8e26a4da None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-10fc1bcb-d1b4-4ad0-a997-dbda8e26a4da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:39.512447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:39.513418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:39.515114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:39.515114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:39.515114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:39.515114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:39.515114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:39.515114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:39.515114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:39.515114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:39.522332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 385/748] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:39.523665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc1bcb-d1b4-4ad0-a997-dbda8e26a4da None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.541976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876791e-6f31-4b31-a9c6-ba31a30027aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.542996 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876791e-6f31-4b31-a9c6-ba31a30027aa None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.543436 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876791e-6f31-4b31-a9c6-ba31a30027aa None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.543957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876791e-6f31-4b31-a9c6-ba31a30027aa None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.546099 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876791e-6f31-4b31-a9c6-ba31a30027aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:39.546793 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876791e-6f31-4b31-a9c6-ba31a30027aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.549358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876791e-6f31-4b31-a9c6-ba31a30027aa None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.583548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 364/749] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Aug 30 01:03:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:39.585102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876791e-6f31-4b31-a9c6-ba31a30027aa None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.645488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc1bcb-d1b4-4ad0-a997-dbda8e26a4da None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.693952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f708d4-378f-401a-8dff-19fe62eeff35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.695364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f708d4-378f-401a-8dff-19fe62eeff35 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.696476 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f708d4-378f-401a-8dff-19fe62eeff35 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.696963 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f708d4-378f-401a-8dff-19fe62eeff35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.712522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f708d4-378f-401a-8dff-19fe62eeff35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.715512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f708d4-378f-401a-8dff-19fe62eeff35 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-53f708d4-378f-401a-8dff-19fe62eeff35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:39.740389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:39.741440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:39.742326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:39.742326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:39.742326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:39.750953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 386/750] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:39.752636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f708d4-378f-401a-8dff-19fe62eeff35 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.778892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876791e-6f31-4b31-a9c6-ba31a30027aa None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.790320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:39.829034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d18854-47a9-499f-a936-a8be3d8b3821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.830023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d18854-47a9-499f-a936-a8be3d8b3821 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.830511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d18854-47a9-499f-a936-a8be3d8b3821 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.831011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d18854-47a9-499f-a936-a8be3d8b3821 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:39.831954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d18854-47a9-499f-a936-a8be3d8b3821 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:39.832498 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d18854-47a9-499f-a936-a8be3d8b3821 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:39.835952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d18854-47a9-499f-a936-a8be3d8b3821 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:39.842855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:39.861906 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 365/751] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:39.863817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d18854-47a9-499f-a936-a8be3d8b3821 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:39.952162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f708d4-378f-401a-8dff-19fe62eeff35 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.996107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d18854-47a9-499f-a936-a8be3d8b3821 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:39.996581 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ceea0-cc3d-4d00-8fa5-ee32642a20bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:39.997225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ceea0-cc3d-4d00-8fa5-ee32642a20bf None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:39.997506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ceea0-cc3d-4d00-8fa5-ee32642a20bf None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:39.997783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ceea0-cc3d-4d00-8fa5-ee32642a20bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.011621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ceea0-cc3d-4d00-8fa5-ee32642a20bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.013278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ceea0-cc3d-4d00-8fa5-ee32642a20bf None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-b68ceea0-cc3d-4d00-8fa5-ee32642a20bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:40.030741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:40.031798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:40.033088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:40.033088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:40.033088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:40.037036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 387/752] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:40.038181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ceea0-cc3d-4d00-8fa5-ee32642a20bf None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.058645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4524d1-afcc-4d2e-903a-8f533b189fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.059686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4524d1-afcc-4d2e-903a-8f533b189fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.060221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4524d1-afcc-4d2e-903a-8f533b189fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.061236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4524d1-afcc-4d2e-903a-8f533b189fb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.062543 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4524d1-afcc-4d2e-903a-8f533b189fb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:40.063085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4524d1-afcc-4d2e-903a-8f533b189fb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.065841 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4524d1-afcc-4d2e-903a-8f533b189fb9 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.089798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 366/753] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:40.092041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c8039-a391-4405-819c-7711c85fae71 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.092527 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c8039-a391-4405-819c-7711c85fae71 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.093040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c8039-a391-4405-819c-7711c85fae71 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.097036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 367/754] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:40.097036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c8039-a391-4405-819c-7711c85fae71 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.142597 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:40.195118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:40.205951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ceea0-cc3d-4d00-8fa5-ee32642a20bf None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.243661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c8039-a391-4405-819c-7711c85fae71 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.249604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5643f-3992-4957-916b-be349d702a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.250223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5643f-3992-4957-916b-be349d702a21 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.250504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5643f-3992-4957-916b-be349d702a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.250778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5643f-3992-4957-916b-be349d702a21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.251509 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5643f-3992-4957-916b-be349d702a21 None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=3330df65b64c4fffbc3bb687d5c51021) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:40.251821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5643f-3992-4957-916b-be349d702a21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.253427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5643f-3992-4957-916b-be349d702a21 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.268090 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 388/755] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:40 2022] GET /identity/v3/role_assignments?user.id=3330df65b64c4fffbc3bb687d5c51021&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:40.268090 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5643f-3992-4957-916b-be349d702a21 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.327271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09bfb5-f2b5-47a0-b25a-74d54fb13e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.329502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09bfb5-f2b5-47a0-b25a-74d54fb13e42 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.330096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09bfb5-f2b5-47a0-b25a-74d54fb13e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.330652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09bfb5-f2b5-47a0-b25a-74d54fb13e42 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.332759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09bfb5-f2b5-47a0-b25a-74d54fb13e42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:40.333839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09bfb5-f2b5-47a0-b25a-74d54fb13e42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.337244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09bfb5-f2b5-47a0-b25a-74d54fb13e42 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.365669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 368/756] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:40.368366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a8cb1-3d98-4811-9a8f-08bdc2834750 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.368652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a8cb1-3d98-4811-9a8f-08bdc2834750 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.368974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a8cb1-3d98-4811-9a8f-08bdc2834750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.399542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-9d4a8cb1-3d98-4811-9a8f-08bdc2834750 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:40.401084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9d4a8cb1-3d98-4811-9a8f-08bdc2834750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:40.431397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5643f-3992-4957-916b-be349d702a21 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.496395 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 369/757] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:03:40.496395 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a8cb1-3d98-4811-9a8f-08bdc2834750 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.515956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:40.517830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086e072-2e56-46bf-91cc-4787a3b86b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.518541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086e072-2e56-46bf-91cc-4787a3b86b01 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.518959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086e072-2e56-46bf-91cc-4787a3b86b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.519323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086e072-2e56-46bf-91cc-4787a3b86b01 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.539785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086e072-2e56-46bf-91cc-4787a3b86b01 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.541663 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086e072-2e56-46bf-91cc-4787a3b86b01 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-d086e072-2e56-46bf-91cc-4787a3b86b01 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:40.564863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:40.566002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:40.566936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:40.566936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:40.566936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:40.566936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:40.566936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:40.566936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:40.566936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:40.566936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:40.566936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:03:40.566936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:40.576861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 389/758] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Aug 30 01:03:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:40.585789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086e072-2e56-46bf-91cc-4787a3b86b01 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.620491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a8cb1-3d98-4811-9a8f-08bdc2834750 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.662706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb1bc0-bab8-4561-b1f4-e9d72748dd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.664127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb1bc0-bab8-4561-b1f4-e9d72748dd09 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.664640 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb1bc0-bab8-4561-b1f4-e9d72748dd09 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.665139 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb1bc0-bab8-4561-b1f4-e9d72748dd09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.674720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fb1bc0-bab8-4561-b1f4-e9d72748dd09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.677274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fb1bc0-bab8-4561-b1f4-e9d72748dd09 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.679489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-a0fb1bc0-bab8-4561-b1f4-e9d72748dd09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:40.692648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:40.693623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:40.694544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:40.694544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:40.694544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:40.694544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:40.694544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:40.694544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:40.694544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:40.694544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:40.698230 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 370/759] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:40.699825 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462d00d-0f95-48c6-a15f-68e0a61c0bdf None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.700364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462d00d-0f95-48c6-a15f-68e0a61c0bdf None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.700838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462d00d-0f95-48c6-a15f-68e0a61c0bdf None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.703858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 371/760] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:40.709158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7c742-409c-4540-9bc0-d783507d1c48 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.709710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7c742-409c-4540-9bc0-d783507d1c48 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.710227 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7c742-409c-4540-9bc0-d783507d1c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.739365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-e9c7c742-409c-4540-9bc0-d783507d1c48 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:40.741332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e9c7c742-409c-4540-9bc0-d783507d1c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:40.805955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086e072-2e56-46bf-91cc-4787a3b86b01 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:40.852315 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc294a-270d-4365-accf-0a8d2d30c772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:40.853064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc294a-270d-4365-accf-0a8d2d30c772 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.853406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc294a-270d-4365-accf-0a8d2d30c772 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.853757 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc294a-270d-4365-accf-0a8d2d30c772 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.854535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc294a-270d-4365-accf-0a8d2d30c772 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:40.854848 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc294a-270d-4365-accf-0a8d2d30c772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:40.856406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc294a-270d-4365-accf-0a8d2d30c772 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:40.868297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 372/761] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:03:40.869003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7c742-409c-4540-9bc0-d783507d1c48 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.884026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 390/762] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Aug 30 01:03:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:40.884026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc294a-270d-4365-accf-0a8d2d30c772 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:40.887722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:40.989673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:41.022629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc294a-270d-4365-accf-0a8d2d30c772 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:41.069074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b639128-6b8f-492d-be3f-739aac0d8311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:41.070620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b639128-6b8f-492d-be3f-739aac0d8311 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:41.071217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b639128-6b8f-492d-be3f-739aac0d8311 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:41.072411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b639128-6b8f-492d-be3f-739aac0d8311 None admin] PATH_INFO: `/v3/services/network` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:41.073549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b639128-6b8f-492d-be3f-739aac0d8311 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:41.075372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b639128-6b8f-492d-be3f-739aac0d8311 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:41.085127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7c742-409c-4540-9bc0-d783507d1c48 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-3b639128-6b8f-492d-be3f-739aac0d8311 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:41.089213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:41.090311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:03:41.091205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:41.096351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 391/763] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:40 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:41.100760 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b639128-6b8f-492d-be3f-739aac0d8311 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:41.162221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ded69-863c-4248-8645-293875b3e08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:41.163279 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ded69-863c-4248-8645-293875b3e08f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:41.163985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ded69-863c-4248-8645-293875b3e08f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:41.165152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ded69-863c-4248-8645-293875b3e08f None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:41.166922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ded69-863c-4248-8645-293875b3e08f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:41.167505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ded69-863c-4248-8645-293875b3e08f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:41.171881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ded69-863c-4248-8645-293875b3e08f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:41.204774 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 373/764] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:41.206164 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ded69-863c-4248-8645-293875b3e08f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:41.241948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b639128-6b8f-492d-be3f-739aac0d8311 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:41.286556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508b997-28bf-4c5c-890f-b7936093ffcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:41.287195 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508b997-28bf-4c5c-890f-b7936093ffcd None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:41.287472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508b997-28bf-4c5c-890f-b7936093ffcd None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:41.287741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508b997-28bf-4c5c-890f-b7936093ffcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:41.303500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:41.317953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:41.367287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ded69-863c-4248-8645-293875b3e08f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:41.390854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508b997-28bf-4c5c-890f-b7936093ffcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:41.392386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508b997-28bf-4c5c-890f-b7936093ffcd None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:41.422758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f764261-840b-46eb-af5a-7dba9392f773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:41.423390 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f764261-840b-46eb-af5a-7dba9392f773 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:41.423789 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f764261-840b-46eb-af5a-7dba9392f773 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:41.423963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f764261-840b-46eb-af5a-7dba9392f773 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:41.424623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f764261-840b-46eb-af5a-7dba9392f773 None admin] RBAC: Adding query filter params (name=network) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:41.424926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f764261-840b-46eb-af5a-7dba9392f773 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:41.426307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f764261-840b-46eb-af5a-7dba9392f773 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:41.433017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 374/765] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:03:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:41.433881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f764261-840b-46eb-af5a-7dba9392f773 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:41.523795 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 392/766] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:03:41.525002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508b997-28bf-4c5c-890f-b7936093ffcd None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:41.590026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f764261-840b-46eb-af5a-7dba9392f773 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:41.593894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:41.626025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:41.632618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e57cc-85af-4cda-84ee-d2b6823267d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:41.633253 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e57cc-85af-4cda-84ee-d2b6823267d1 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:41.633531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e57cc-85af-4cda-84ee-d2b6823267d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:41.633800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e57cc-85af-4cda-84ee-d2b6823267d1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:41.634384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e57cc-85af-4cda-84ee-d2b6823267d1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:41.635781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e57cc-85af-4cda-84ee-d2b6823267d1 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-ea5e57cc-85af-4cda-84ee-d2b6823267d1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:41.648100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:41.649231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:03:41.650070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:41.654832 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 375/767] 10.222.0.44 () {56 vars in 1129 bytes} [Tue Aug 30 01:03:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:41.656059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e57cc-85af-4cda-84ee-d2b6823267d1 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:41.701512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508b997-28bf-4c5c-890f-b7936093ffcd None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:41.754559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133ea06-c80d-45b0-a560-c78b6adcbf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:41.755758 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133ea06-c80d-45b0-a560-c78b6adcbf5d None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:41.756320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133ea06-c80d-45b0-a560-c78b6adcbf5d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:41.757040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133ea06-c80d-45b0-a560-c78b6adcbf5d None admin] PATH_INFO: `/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/049399a752ed4646a13f8e4a3a62d8e9/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:41.759865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133ea06-c80d-45b0-a560-c78b6adcbf5d None admin] RBAC: Authorizing `identity:create_grant(project_id=9188023bf51d4737b801e3fbd9af95b1, user_id=049399a752ed4646a13f8e4a3a62d8e9, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:41.761821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133ea06-c80d-45b0-a560-c78b6adcbf5d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:41.781979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 393/768] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:03:41 2022] PUT /identity/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/049399a752ed4646a13f8e4a3a62d8e9/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:41.783169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133ea06-c80d-45b0-a560-c78b6adcbf5d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:41.848395 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e57cc-85af-4cda-84ee-d2b6823267d1 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:41.924434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49eb18a-7450-4a08-af71-e3046fe4865c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:41.925424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49eb18a-7450-4a08-af71-e3046fe4865c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:41.925870 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49eb18a-7450-4a08-af71-e3046fe4865c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:41.926510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49eb18a-7450-4a08-af71-e3046fe4865c None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:41.927862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49eb18a-7450-4a08-af71-e3046fe4865c None admin] RBAC: Adding query filter params (type=network) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:41.928587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49eb18a-7450-4a08-af71-e3046fe4865c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:41.930787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49eb18a-7450-4a08-af71-e3046fe4865c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:41.933458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:41.938853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133ea06-c80d-45b0-a560-c78b6adcbf5d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:41.949587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 376/769] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:03:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:41.950842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49eb18a-7450-4a08-af71-e3046fe4865c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:41.991749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8532c05-ee4b-44ce-9302-fedf06170ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:41.992562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8532c05-ee4b-44ce-9302-fedf06170ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:41.992843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8532c05-ee4b-44ce-9302-fedf06170ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:41.993111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8532c05-ee4b-44ce-9302-fedf06170ef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:42.010919 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8532c05-ee4b-44ce-9302-fedf06170ef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:42.012723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8532c05-ee4b-44ce-9302-fedf06170ef8 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-e8532c05-ee4b-44ce-9302-fedf06170ef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:42.042322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:42.043789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:42.045096 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:42.045096 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:42.045096 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:42.045849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 394/770] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:42.047411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8532c05-ee4b-44ce-9302-fedf06170ef8 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:42.054675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:42.124458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49eb18a-7450-4a08-af71-e3046fe4865c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:42.185028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32274dde-6d08-4614-af58-cfc60215b95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:42.185960 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32274dde-6d08-4614-af58-cfc60215b95b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:42.186351 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32274dde-6d08-4614-af58-cfc60215b95b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:42.186672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32274dde-6d08-4614-af58-cfc60215b95b None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:42.188263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32274dde-6d08-4614-af58-cfc60215b95b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:42.188913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32274dde-6d08-4614-af58-cfc60215b95b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:42.190758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32274dde-6d08-4614-af58-cfc60215b95b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:42.208270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 377/771] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Aug 30 01:03:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:42.210100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32274dde-6d08-4614-af58-cfc60215b95b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:42.238897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8532c05-ee4b-44ce-9302-fedf06170ef8 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:42.301454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be205826-51f3-4411-9997-b92cefad11d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:42.302361 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be205826-51f3-4411-9997-b92cefad11d0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:42.302785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be205826-51f3-4411-9997-b92cefad11d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:42.303235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be205826-51f3-4411-9997-b92cefad11d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:42.304298 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be205826-51f3-4411-9997-b92cefad11d0 None admin] RBAC: Adding query filter params (interface=public, service_id=d4b7b923d1b24313983b30d8ad14a59c, region_id=RegionOne) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:42.304795 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be205826-51f3-4411-9997-b92cefad11d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:42.306643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be205826-51f3-4411-9997-b92cefad11d0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:42.321202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 395/772] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Aug 30 01:03:42 2022] GET /identity/v3/endpoints?service_id=d4b7b923d1b24313983b30d8ad14a59c&interface=public®ion_id=RegionOne => generated 194 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:42.322981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be205826-51f3-4411-9997-b92cefad11d0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:42.372733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:42.437191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32274dde-6d08-4614-af58-cfc60215b95b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:42.471174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:42.493799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08daea46-493f-46f8-803c-bb5e43eaa9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:42.495012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daea46-493f-46f8-803c-bb5e43eaa9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:42.495520 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daea46-493f-46f8-803c-bb5e43eaa9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:42.495987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daea46-493f-46f8-803c-bb5e43eaa9a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:42.497375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08daea46-493f-46f8-803c-bb5e43eaa9a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:42.498161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08daea46-493f-46f8-803c-bb5e43eaa9a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:42.500385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08daea46-493f-46f8-803c-bb5e43eaa9a6 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:42.501263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be205826-51f3-4411-9997-b92cefad11d0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:42.527287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 378/773] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:42.528990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08daea46-493f-46f8-803c-bb5e43eaa9a6 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:42.542799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a584d-afe4-49bd-8c52-f90712b453af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:42.543756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a584d-afe4-49bd-8c52-f90712b453af None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:42.544577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a584d-afe4-49bd-8c52-f90712b453af None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:42.545629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a584d-afe4-49bd-8c52-f90712b453af None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:42.546461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a584d-afe4-49bd-8c52-f90712b453af None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:42.546910 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a584d-afe4-49bd-8c52-f90712b453af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:42.548624 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a584d-afe4-49bd-8c52-f90712b453af None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:42.555839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 396/774] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Aug 30 01:03:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:42.556918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a584d-afe4-49bd-8c52-f90712b453af None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:42.678188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:42.709819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a584d-afe4-49bd-8c52-f90712b453af None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:42.724176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08daea46-493f-46f8-803c-bb5e43eaa9a6 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:42.763149 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8e8f3-78a1-45f0-b93d-a47f56d700d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:42.764367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8e8f3-78a1-45f0-b93d-a47f56d700d5 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:42.764588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8e8f3-78a1-45f0-b93d-a47f56d700d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:42.764828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8e8f3-78a1-45f0-b93d-a47f56d700d5 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:42.765432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8e8f3-78a1-45f0-b93d-a47f56d700d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:42.766904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8e8f3-78a1-45f0-b93d-a47f56d700d5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:42.768975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588df625-81e0-47c1-948e-297460e454a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:42.769894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588df625-81e0-47c1-948e-297460e454a9 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:42.770185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588df625-81e0-47c1-948e-297460e454a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:42.770520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588df625-81e0-47c1-948e-297460e454a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:42.776047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 379/775] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Aug 30 01:03:42 2022] GET /identity/v3/services => generated 772 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:42.776047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8e8f3-78a1-45f0-b93d-a47f56d700d5 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:42.798255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588df625-81e0-47c1-948e-297460e454a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:42.799529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588df625-81e0-47c1-948e-297460e454a9 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-588df625-81e0-47c1-948e-297460e454a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:42.813401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:42.814359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:42.815452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:42.815452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:42.815452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:42.815452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:42.815452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:42.815452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:42.815452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:42.815452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:42.821709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 397/776] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:42.823622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec954ff1-0fbb-457e-851b-8b6090b57ee8 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:42.824130 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec954ff1-0fbb-457e-851b-8b6090b57ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:42.824600 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec954ff1-0fbb-457e-851b-8b6090b57ee8 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:42.828576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 398/777] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:42.829989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec954ff1-0fbb-457e-851b-8b6090b57ee8 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:42.862446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:42.932669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8e8f3-78a1-45f0-b93d-a47f56d700d5 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:42.987431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec954ff1-0fbb-457e-851b-8b6090b57ee8 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:42.990179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbb893-5f58-4052-a4d5-e3970dc9eb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:42.991229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbb893-5f58-4052-a4d5-e3970dc9eb8d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:42.991782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbb893-5f58-4052-a4d5-e3970dc9eb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:42.992914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbb893-5f58-4052-a4d5-e3970dc9eb8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:42.993670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbb893-5f58-4052-a4d5-e3970dc9eb8d None admin] RBAC: Adding query filter params (interface=public, service_id=93ecdf9934bc436ba633562de33f1da1, region_id=RegionOne) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:42.994219 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbb893-5f58-4052-a4d5-e3970dc9eb8d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:42.996910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbb893-5f58-4052-a4d5-e3970dc9eb8d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:43.016023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 380/778] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Aug 30 01:03:42 2022] GET /identity/v3/endpoints?service_id=93ecdf9934bc436ba633562de33f1da1&interface=public®ion_id=RegionOne => generated 194 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:43.016023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6281ec-3dd1-4694-a974-064992761b1c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:43.016023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6281ec-3dd1-4694-a974-064992761b1c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:43.016558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6281ec-3dd1-4694-a974-064992761b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:43.031372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483896d-10a3-4364-81e6-b4560bbecc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:43.032014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483896d-10a3-4364-81e6-b4560bbecc19 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:43.032312 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483896d-10a3-4364-81e6-b4560bbecc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:43.032582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483896d-10a3-4364-81e6-b4560bbecc19 None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:43.033311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483896d-10a3-4364-81e6-b4560bbecc19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:43.033618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483896d-10a3-4364-81e6-b4560bbecc19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:43.035172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483896d-10a3-4364-81e6-b4560bbecc19 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:43.042970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-6e6281ec-3dd1-4694-a974-064992761b1c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:43.044685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6e6281ec-3dd1-4694-a974-064992761b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:43.049932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:43.053084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 399/779] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:43.054374 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483896d-10a3-4364-81e6-b4560bbecc19 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:43.163449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 381/780] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:03:43.164635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6281ec-3dd1-4694-a974-064992761b1c None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:43.181866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483896d-10a3-4364-81e6-b4560bbecc19 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:43.268567 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:43.278127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f957fa2-0e0a-4b78-ada1-247c59933511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:43.280875 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f957fa2-0e0a-4b78-ada1-247c59933511 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:43.281451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f957fa2-0e0a-4b78-ada1-247c59933511 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:43.282010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f957fa2-0e0a-4b78-ada1-247c59933511 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:43.283113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f957fa2-0e0a-4b78-ada1-247c59933511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:43.285785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f957fa2-0e0a-4b78-ada1-247c59933511 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:43.298269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 400/781] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Aug 30 01:03:43 2022] GET /identity/v3/services => generated 772 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:43.388453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6281ec-3dd1-4694-a974-064992761b1c None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:43.446859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3201c26-53ca-449e-8e98-58baccc764b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:43.447611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3201c26-53ca-449e-8e98-58baccc764b3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:43.448043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3201c26-53ca-449e-8e98-58baccc764b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:43.448328 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3201c26-53ca-449e-8e98-58baccc764b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:43.449152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3201c26-53ca-449e-8e98-58baccc764b3 None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=8a3c3027b61d4f75a67e0a1bb8b52903) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:43.449597 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3201c26-53ca-449e-8e98-58baccc764b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:43.451141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3201c26-53ca-449e-8e98-58baccc764b3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:43.461790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 382/782] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:43 2022] GET /identity/v3/role_assignments?user.id=8a3c3027b61d4f75a67e0a1bb8b52903&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:43.611170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f957fa2-0e0a-4b78-ada1-247c59933511 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:43.715880 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:43.776992 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f957fa2-0e0a-4b78-ada1-247c59933511 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:43.833213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f1ade-fdbc-42ca-8818-e5e890ef168b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:43.833867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f1ade-fdbc-42ca-8818-e5e890ef168b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:43.834150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f1ade-fdbc-42ca-8818-e5e890ef168b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:43.834425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f1ade-fdbc-42ca-8818-e5e890ef168b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:43.858077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f1ade-fdbc-42ca-8818-e5e890ef168b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-4f0f1ade-fdbc-42ca-8818-e5e890ef168b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:43.858505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:43.859518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:43.860489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 401/783] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:43.866356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3201c26-53ca-449e-8e98-58baccc764b3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:44.041061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3201c26-53ca-449e-8e98-58baccc764b3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:44.051178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:44.099785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d9b25-6748-4cad-864c-5fb53011e99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:44.100451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d9b25-6748-4cad-864c-5fb53011e99c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:44.100736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d9b25-6748-4cad-864c-5fb53011e99c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:44.101025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d9b25-6748-4cad-864c-5fb53011e99c None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:44.101691 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d9b25-6748-4cad-864c-5fb53011e99c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:44.101998 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d9b25-6748-4cad-864c-5fb53011e99c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:44.103466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d9b25-6748-4cad-864c-5fb53011e99c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:44.118750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 383/784] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:44.125335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f1ade-fdbc-42ca-8818-e5e890ef168b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:44.285438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860282-8d0b-4e17-b543-8478ad38e3ab None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:44.285841 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860282-8d0b-4e17-b543-8478ad38e3ab None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:44.286017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860282-8d0b-4e17-b543-8478ad38e3ab None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:44.287665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 384/785] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:44.294394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e296ff-ffa8-4c9c-b1f0-9e24b1060136 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:44.294670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e296ff-ffa8-4c9c-b1f0-9e24b1060136 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:44.295563 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e296ff-ffa8-4c9c-b1f0-9e24b1060136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:44.314941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f1ade-fdbc-42ca-8818-e5e890ef168b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:44.351544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-b5e296ff-ffa8-4c9c-b1f0-9e24b1060136 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:44.353021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b5e296ff-ffa8-4c9c-b1f0-9e24b1060136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:44.367327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:44.382075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812b275-e940-48fc-bb8d-c7ad4259b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:44.383061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812b275-e940-48fc-bb8d-c7ad4259b6ce None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:44.383523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812b275-e940-48fc-bb8d-c7ad4259b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:44.383941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812b275-e940-48fc-bb8d-c7ad4259b6ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:44.398060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812b275-e940-48fc-bb8d-c7ad4259b6ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:44.399625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812b275-e940-48fc-bb8d-c7ad4259b6ce None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-1812b275-e940-48fc-bb8d-c7ad4259b6ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:44.416126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:44.417231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:44.418276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:44.418276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:44.418276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:44.418276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 402/786] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:44.427820 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812b275-e940-48fc-bb8d-c7ad4259b6ce None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:44.534816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 385/787] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:03:44.623102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812b275-e940-48fc-bb8d-c7ad4259b6ce None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:44.680062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b5b7d-8bad-4194-9426-ab94b1596edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:44.681087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b5b7d-8bad-4194-9426-ab94b1596edf None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:44.681552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b5b7d-8bad-4194-9426-ab94b1596edf None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:44.682035 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b5b7d-8bad-4194-9426-ab94b1596edf None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:44.683221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b5b7d-8bad-4194-9426-ab94b1596edf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:44.683769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b5b7d-8bad-4194-9426-ab94b1596edf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:44.688744 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b5b7d-8bad-4194-9426-ab94b1596edf None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:44.706147 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 403/788] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:44.711883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e296ff-ffa8-4c9c-b1f0-9e24b1060136 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:45.007726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b5b7d-8bad-4194-9426-ab94b1596edf None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:45.040469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:45.053401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e296ff-ffa8-4c9c-b1f0-9e24b1060136 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:45.104461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:45.152487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2b5fa-ac63-4615-944d-4825c4523134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:45.153149 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2b5fa-ac63-4615-944d-4825c4523134 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.153428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2b5fa-ac63-4615-944d-4825c4523134 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.153698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2b5fa-ac63-4615-944d-4825c4523134 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.172040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2b5fa-ac63-4615-944d-4825c4523134 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:45.173709 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2b5fa-ac63-4615-944d-4825c4523134 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:45.181473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b5b7d-8bad-4194-9426-ab94b1596edf None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-11a2b5fa-ac63-4615-944d-4825c4523134 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:45.198677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:45.199680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:45.200683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:45.200683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:45.200683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:45.200683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:45.200683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:45.200683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:45.200683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:45.200683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:45.200683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:03:45.200683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:45.200683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 386/789] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 489 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:45.206364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2b5fa-ac63-4615-944d-4825c4523134 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:45.221819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ef365-51a6-42f2-9c96-9a895bdee3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:45.222554 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef365-51a6-42f2-9c96-9a895bdee3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.222808 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef365-51a6-42f2-9c96-9a895bdee3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.223089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef365-51a6-42f2-9c96-9a895bdee3d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.246708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ef365-51a6-42f2-9c96-9a895bdee3d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-2e1ef365-51a6-42f2-9c96-9a895bdee3d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:45.255728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:45.258458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:45.261643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 404/790] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:45.268041 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ef365-51a6-42f2-9c96-9a895bdee3d3 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:45.395860 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:45.403259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ef365-51a6-42f2-9c96-9a895bdee3d3 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:45.443689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c2371-ac98-4027-8077-36331aa9b00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:45.444372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c2371-ac98-4027-8077-36331aa9b00f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.444657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c2371-ac98-4027-8077-36331aa9b00f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.444949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c2371-ac98-4027-8077-36331aa9b00f None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.445694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c2371-ac98-4027-8077-36331aa9b00f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:45.446039 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c2371-ac98-4027-8077-36331aa9b00f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:45.446355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2b5fa-ac63-4615-944d-4825c4523134 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:45.447517 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c2371-ac98-4027-8077-36331aa9b00f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:45.458114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 405/791] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:45.458849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbdff4-be1e-46ba-b864-38dd494dd0a1 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.459129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbdff4-be1e-46ba-b864-38dd494dd0a1 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.459397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbdff4-be1e-46ba-b864-38dd494dd0a1 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.461099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 406/792] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:45.472023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbdff4-be1e-46ba-b864-38dd494dd0a1 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:45.525690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef988d-49dd-40dc-af15-7478972d247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:45.527247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef988d-49dd-40dc-af15-7478972d247e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.528205 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef988d-49dd-40dc-af15-7478972d247e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.528793 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef988d-49dd-40dc-af15-7478972d247e None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.531694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef988d-49dd-40dc-af15-7478972d247e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:45.532401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef988d-49dd-40dc-af15-7478972d247e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:45.535930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef988d-49dd-40dc-af15-7478972d247e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:45.580819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 387/793] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Aug 30 01:03:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:45.584024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1640a4-b24d-4095-b3a8-7675ee947d9a None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.584502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1640a4-b24d-4095-b3a8-7675ee947d9a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.584875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1640a4-b24d-4095-b3a8-7675ee947d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.597079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbdff4-be1e-46ba-b864-38dd494dd0a1 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:45.620196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-ec1640a4-b24d-4095-b3a8-7675ee947d9a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:45.622016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ec1640a4-b24d-4095-b3a8-7675ee947d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:45.635851 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:45.646924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66ba42-8d4b-4515-b956-42566bc137eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:45.647713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66ba42-8d4b-4515-b956-42566bc137eb None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.648079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66ba42-8d4b-4515-b956-42566bc137eb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.648551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66ba42-8d4b-4515-b956-42566bc137eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.663534 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66ba42-8d4b-4515-b956-42566bc137eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:45.665426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66ba42-8d4b-4515-b956-42566bc137eb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-af66ba42-8d4b-4515-b956-42566bc137eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:45.691021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:45.693047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:45.694911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:45.694911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:45.694911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:45.694911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 407/794] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:45.694911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66ba42-8d4b-4515-b956-42566bc137eb None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:45.717566 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:45.754452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 388/795] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:03:45.757469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1640a4-b24d-4095-b3a8-7675ee947d9a None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:45.922566 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66ba42-8d4b-4515-b956-42566bc137eb None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:45.969483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85908c5b-b0e6-4f5c-83b0-6920f5340974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:45.970577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85908c5b-b0e6-4f5c-83b0-6920f5340974 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:45.971144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85908c5b-b0e6-4f5c-83b0-6920f5340974 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:45.971719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85908c5b-b0e6-4f5c-83b0-6920f5340974 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:45.986640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85908c5b-b0e6-4f5c-83b0-6920f5340974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:45.988845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85908c5b-b0e6-4f5c-83b0-6920f5340974 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-85908c5b-b0e6-4f5c-83b0-6920f5340974 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:46.007142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:46.008268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:46.009503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:46.009503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:46.009503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:46.011372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 408/796] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:46.019345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85908c5b-b0e6-4f5c-83b0-6920f5340974 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.047607 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1640a4-b24d-4095-b3a8-7675ee947d9a None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:46.120405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d864501-b8c2-4b91-b474-aa1fd48de2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:46.121097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d864501-b8c2-4b91-b474-aa1fd48de2fc None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.121301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d864501-b8c2-4b91-b474-aa1fd48de2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.121575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d864501-b8c2-4b91-b474-aa1fd48de2fc None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.122251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d864501-b8c2-4b91-b474-aa1fd48de2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:46.122567 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d864501-b8c2-4b91-b474-aa1fd48de2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:46.124835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d864501-b8c2-4b91-b474-aa1fd48de2fc None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.148833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 389/797] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:46.153161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d864501-b8c2-4b91-b474-aa1fd48de2fc None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.159292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:46.161094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:46.227477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85908c5b-b0e6-4f5c-83b0-6920f5340974 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:46.286062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f483-c6c7-457c-bd66-eae777d9b00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:46.286690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f483-c6c7-457c-bd66-eae777d9b00f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.286969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f483-c6c7-457c-bd66-eae777d9b00f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.287219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f483-c6c7-457c-bd66-eae777d9b00f None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.287892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f483-c6c7-457c-bd66-eae777d9b00f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:46.288234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f483-c6c7-457c-bd66-eae777d9b00f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:46.289660 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f483-c6c7-457c-bd66-eae777d9b00f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.304950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 409/798] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:46.306016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f483-c6c7-457c-bd66-eae777d9b00f None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.313122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d864501-b8c2-4b91-b474-aa1fd48de2fc None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:46.373771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bcdbb-a9e6-4eec-b421-238e98e6163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:46.374637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bcdbb-a9e6-4eec-b421-238e98e6163b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.374902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bcdbb-a9e6-4eec-b421-238e98e6163b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.375177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bcdbb-a9e6-4eec-b421-238e98e6163b None admin] PATH_INFO: `/v3/services/network` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.375864 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bcdbb-a9e6-4eec-b421-238e98e6163b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:46.377411 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bcdbb-a9e6-4eec-b421-238e98e6163b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-855bcdbb-a9e6-4eec-b421-238e98e6163b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:46.387492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:46.389407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:46.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 390/799] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:46 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:46.391941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8660664-b983-46cd-8b09-281792b82fce None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.391941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8660664-b983-46cd-8b09-281792b82fce None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.391941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8660664-b983-46cd-8b09-281792b82fce None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.393291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 391/800] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:46.394175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8660664-b983-46cd-8b09-281792b82fce None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.438110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f483-c6c7-457c-bd66-eae777d9b00f None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:46.466005 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:46.483858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d75700-70f6-47b9-a734-9db09d64312c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:46.485855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d75700-70f6-47b9-a734-9db09d64312c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.485855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d75700-70f6-47b9-a734-9db09d64312c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.485855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d75700-70f6-47b9-a734-9db09d64312c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.493118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:46.495112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d75700-70f6-47b9-a734-9db09d64312c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:46.497165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d75700-70f6-47b9-a734-9db09d64312c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-a1d75700-70f6-47b9-a734-9db09d64312c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:46.513100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:46.515125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:46.517135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:46.517135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:46.517135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:46.517135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:46.517135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:46.517135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:46.517135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:46.517135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:46.517135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:03:46.517135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:46.517135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 410/801] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Aug 30 01:03:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:46.518811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f16b-7f7c-4552-9038-79e8fb737584 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.519229 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f16b-7f7c-4552-9038-79e8fb737584 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.519672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f16b-7f7c-4552-9038-79e8fb737584 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.524155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 411/802] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:46.524155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456f16b-7f7c-4552-9038-79e8fb737584 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.543405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8660664-b983-46cd-8b09-281792b82fce None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:46.590382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6322733-2fdf-43ea-af76-7241e0f32d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:46.591578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6322733-2fdf-43ea-af76-7241e0f32d70 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.592918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6322733-2fdf-43ea-af76-7241e0f32d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.595403 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6322733-2fdf-43ea-af76-7241e0f32d70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.605808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6322733-2fdf-43ea-af76-7241e0f32d70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:46.608443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6322733-2fdf-43ea-af76-7241e0f32d70 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-b6322733-2fdf-43ea-af76-7241e0f32d70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:46.622198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:46.624801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:46.626318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:46.626318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:46.626318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:46.626318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:46.626318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:46.626318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:46.626318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:46.626318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:46.626318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 392/803] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:46.626318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895fc33-fbf4-46ea-87cd-d0af22f6cadb None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.626318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895fc33-fbf4-46ea-87cd-d0af22f6cadb None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.627103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895fc33-fbf4-46ea-87cd-d0af22f6cadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.647850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8895fc33-fbf4-46ea-87cd-d0af22f6cadb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:46.649236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8895fc33-fbf4-46ea-87cd-d0af22f6cadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:46.693123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456f16b-7f7c-4552-9038-79e8fb737584 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:46.762460 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2df389-5413-4887-96da-f953ceda54be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:46.762999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2df389-5413-4887-96da-f953ceda54be None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.763287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2df389-5413-4887-96da-f953ceda54be None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.763774 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2df389-5413-4887-96da-f953ceda54be None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.764898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2df389-5413-4887-96da-f953ceda54be None admin] RBAC: Adding query filter params (name=network) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:46.765324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2df389-5413-4887-96da-f953ceda54be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:46.767527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2df389-5413-4887-96da-f953ceda54be None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:46.772977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 393/804] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:03:46.774175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895fc33-fbf4-46ea-87cd-d0af22f6cadb None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.781520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 412/805] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:03:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:46.781520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76538980-ce86-4ed5-8000-ef4d693fb2b8 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.781520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76538980-ce86-4ed5-8000-ef4d693fb2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.781520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76538980-ce86-4ed5-8000-ef4d693fb2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.820681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-76538980-ce86-4ed5-8000-ef4d693fb2b8 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:46.822932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-76538980-ce86-4ed5-8000-ef4d693fb2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:46.885377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:46.889208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:46.937934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 413/806] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:03:46.949619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76538980-ce86-4ed5-8000-ef4d693fb2b8 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:46.967818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895fc33-fbf4-46ea-87cd-d0af22f6cadb None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:47.049163 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdc9fb-324f-452c-8d1f-4e63a7813ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:47.049900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdc9fb-324f-452c-8d1f-4e63a7813ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:47.051748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdc9fb-324f-452c-8d1f-4e63a7813ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:47.052225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdc9fb-324f-452c-8d1f-4e63a7813ce5 None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:47.053196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdc9fb-324f-452c-8d1f-4e63a7813ce5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:47.053806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdc9fb-324f-452c-8d1f-4e63a7813ce5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:47.055811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdc9fb-324f-452c-8d1f-4e63a7813ce5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:47.096392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 394/807] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Aug 30 01:03:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:47.097413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdc9fb-324f-452c-8d1f-4e63a7813ce5 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:47.153748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76538980-ce86-4ed5-8000-ef4d693fb2b8 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:47.223731 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8ef50-2da8-4ddf-b7dc-2ed87cd7bc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:47.224536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8ef50-2da8-4ddf-b7dc-2ed87cd7bc4b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:47.224816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8ef50-2da8-4ddf-b7dc-2ed87cd7bc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:47.225085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8ef50-2da8-4ddf-b7dc-2ed87cd7bc4b None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:47.225790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8ef50-2da8-4ddf-b7dc-2ed87cd7bc4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:47.226200 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8ef50-2da8-4ddf-b7dc-2ed87cd7bc4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:47.228261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8ef50-2da8-4ddf-b7dc-2ed87cd7bc4b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:47.242752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 414/808] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:47.248680 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8ef50-2da8-4ddf-b7dc-2ed87cd7bc4b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:47.255166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdc9fb-324f-452c-8d1f-4e63a7813ce5 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:47.289892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:47.305478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:47.324079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae25c5e-61cb-42a6-9e49-aba59478ddd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:47.326384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae25c5e-61cb-42a6-9e49-aba59478ddd5 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:47.326384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae25c5e-61cb-42a6-9e49-aba59478ddd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:47.326956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae25c5e-61cb-42a6-9e49-aba59478ddd5 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:47.327769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae25c5e-61cb-42a6-9e49-aba59478ddd5 None admin] RBAC: Adding query filter params (type=network) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:47.328235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae25c5e-61cb-42a6-9e49-aba59478ddd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:47.330473 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae25c5e-61cb-42a6-9e49-aba59478ddd5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:47.339855 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 395/809] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:03:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:47.341593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae25c5e-61cb-42a6-9e49-aba59478ddd5 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:47.401819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8ef50-2da8-4ddf-b7dc-2ed87cd7bc4b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:47.467504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55639e05-e632-418f-a829-a1745967d941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:47.470040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55639e05-e632-418f-a829-a1745967d941 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:47.470705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55639e05-e632-418f-a829-a1745967d941 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:47.471186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55639e05-e632-418f-a829-a1745967d941 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:47.493162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55639e05-e632-418f-a829-a1745967d941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-55639e05-e632-418f-a829-a1745967d941 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:47.505956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:47.507737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 415/810] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:47.512786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55639e05-e632-418f-a829-a1745967d941 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:47.518698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae25c5e-61cb-42a6-9e49-aba59478ddd5 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:47.573293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7345b-53ef-4514-b9d9-c135169aa7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:47.573923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7345b-53ef-4514-b9d9-c135169aa7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:47.574488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7345b-53ef-4514-b9d9-c135169aa7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:47.574767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7345b-53ef-4514-b9d9-c135169aa7d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:47.642799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:47.666568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7345b-53ef-4514-b9d9-c135169aa7d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:47.668479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7345b-53ef-4514-b9d9-c135169aa7d8 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:47.830225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 396/811] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:03:47.830995 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:47.831352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7345b-53ef-4514-b9d9-c135169aa7d8 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:47.840821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55639e05-e632-418f-a829-a1745967d941 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:47.889149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e10a8-1183-4d96-a7e0-ead45bea9a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:47.889928 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e10a8-1183-4d96-a7e0-ead45bea9a6b None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:47.890248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e10a8-1183-4d96-a7e0-ead45bea9a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:47.890469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e10a8-1183-4d96-a7e0-ead45bea9a6b None admin] PATH_INFO: `/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/3330df65b64c4fffbc3bb687d5c51021/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:47.892706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e10a8-1183-4d96-a7e0-ead45bea9a6b None admin] RBAC: Authorizing `identity:create_grant(project_id=9188023bf51d4737b801e3fbd9af95b1, user_id=3330df65b64c4fffbc3bb687d5c51021, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:47.894259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e10a8-1183-4d96-a7e0-ead45bea9a6b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:47.910505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 416/812] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:03:47 2022] PUT /identity/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/3330df65b64c4fffbc3bb687d5c51021/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:47.911882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e10a8-1183-4d96-a7e0-ead45bea9a6b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:47.987955 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:48.036804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7345b-53ef-4514-b9d9-c135169aa7d8 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:48.065486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e10a8-1183-4d96-a7e0-ead45bea9a6b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:48.106015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c694c0c-aafe-4dff-9d4d-da63ae6b43ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:48.106864 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c694c0c-aafe-4dff-9d4d-da63ae6b43ee None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:48.107972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c694c0c-aafe-4dff-9d4d-da63ae6b43ee None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:48.108480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c694c0c-aafe-4dff-9d4d-da63ae6b43ee None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:48.109512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c694c0c-aafe-4dff-9d4d-da63ae6b43ee None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:48.111838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c694c0c-aafe-4dff-9d4d-da63ae6b43ee None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-5c694c0c-aafe-4dff-9d4d-da63ae6b43ee None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:48.126485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:48.128649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:48.130639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 397/813] 10.222.0.44 () {56 vars in 1129 bytes} [Tue Aug 30 01:03:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:48.132425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c694c0c-aafe-4dff-9d4d-da63ae6b43ee None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:48.135288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e14a69-1d82-4b18-8c49-7e91710c617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:48.137989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e14a69-1d82-4b18-8c49-7e91710c617d None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:48.138632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e14a69-1d82-4b18-8c49-7e91710c617d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:48.139201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e14a69-1d82-4b18-8c49-7e91710c617d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:48.140376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e14a69-1d82-4b18-8c49-7e91710c617d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:48.142547 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e14a69-1d82-4b18-8c49-7e91710c617d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:48.187053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 417/814] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:47 2022] POST /identity/v3/endpoints => generated 337 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:48.188640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e14a69-1d82-4b18-8c49-7e91710c617d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:48.209020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:48.351933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c694c0c-aafe-4dff-9d4d-da63ae6b43ee None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:48.399685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e14a69-1d82-4b18-8c49-7e91710c617d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:48.412054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ded73b-9d98-46d7-ab2c-a1011682f36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:48.413139 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ded73b-9d98-46d7-ab2c-a1011682f36b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:48.413756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ded73b-9d98-46d7-ab2c-a1011682f36b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:48.414970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ded73b-9d98-46d7-ab2c-a1011682f36b None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:48.416248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ded73b-9d98-46d7-ab2c-a1011682f36b None admin] RBAC: Adding query filter params (name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:48.416886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ded73b-9d98-46d7-ab2c-a1011682f36b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:48.419694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ded73b-9d98-46d7-ab2c-a1011682f36b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:48.433910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 398/815] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:48.435816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ded73b-9d98-46d7-ab2c-a1011682f36b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:48.450299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:48.470673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67af90-d9e1-463a-8747-4e10b1a101c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:48.471548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67af90-d9e1-463a-8747-4e10b1a101c6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:48.471788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67af90-d9e1-463a-8747-4e10b1a101c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:48.472127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67af90-d9e1-463a-8747-4e10b1a101c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:48.486067 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67af90-d9e1-463a-8747-4e10b1a101c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:48.487568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67af90-d9e1-463a-8747-4e10b1a101c6 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-5d67af90-d9e1-463a-8747-4e10b1a101c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:48.509356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:48.510394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:48.511294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:48.511294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:48.511294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:48.511294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 418/816] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:48.511745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67af90-d9e1-463a-8747-4e10b1a101c6 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:48.599571 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ded73b-9d98-46d7-ab2c-a1011682f36b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:48.607194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:48.650979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9a08-4e91-4312-849d-b421e17b066f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:48.652494 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9a08-4e91-4312-849d-b421e17b066f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:48.652781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9a08-4e91-4312-849d-b421e17b066f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:48.653055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9a08-4e91-4312-849d-b421e17b066f None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:48.654081 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec9a08-4e91-4312-849d-b421e17b066f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:48.654472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec9a08-4e91-4312-849d-b421e17b066f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:48.656969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec9a08-4e91-4312-849d-b421e17b066f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:48.674059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 399/817] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Aug 30 01:03:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:48.674059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9a08-4e91-4312-849d-b421e17b066f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:48.674463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67af90-d9e1-463a-8747-4e10b1a101c6 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:48.737671 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a66d4-87ca-49fc-aff0-3ceca6983582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:48.738731 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a66d4-87ca-49fc-aff0-3ceca6983582 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:48.739382 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a66d4-87ca-49fc-aff0-3ceca6983582 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:48.739854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a66d4-87ca-49fc-aff0-3ceca6983582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:48.755935 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a66d4-87ca-49fc-aff0-3ceca6983582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:48.758831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a66d4-87ca-49fc-aff0-3ceca6983582 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-bd6a66d4-87ca-49fc-aff0-3ceca6983582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:48.785447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:48.787022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:48.788501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:48.788501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:48.788501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:48.788501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 419/818] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:48.788900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a66d4-87ca-49fc-aff0-3ceca6983582 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:48.806852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:48.855035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9a08-4e91-4312-849d-b421e17b066f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:48.908137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e2a5c-c859-440a-8387-f98040cec755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:48.909503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e2a5c-c859-440a-8387-f98040cec755 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:48.910033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e2a5c-c859-440a-8387-f98040cec755 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:48.910344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e2a5c-c859-440a-8387-f98040cec755 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:48.911176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e2a5c-c859-440a-8387-f98040cec755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:48.911483 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e2a5c-c859-440a-8387-f98040cec755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:48.912905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e2a5c-c859-440a-8387-f98040cec755 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:48.938818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 400/819] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:48.938818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e2a5c-c859-440a-8387-f98040cec755 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:48.964396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a66d4-87ca-49fc-aff0-3ceca6983582 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:48.985524 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:49.017313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e560cf1-84ed-4fe5-aa7b-509e0788e0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:49.017913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e560cf1-84ed-4fe5-aa7b-509e0788e0ce None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.018352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e560cf1-84ed-4fe5-aa7b-509e0788e0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.018660 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e560cf1-84ed-4fe5-aa7b-509e0788e0ce None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.019599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e560cf1-84ed-4fe5-aa7b-509e0788e0ce None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:49.019898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e560cf1-84ed-4fe5-aa7b-509e0788e0ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:49.021489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e560cf1-84ed-4fe5-aa7b-509e0788e0ce None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:49.034368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 420/820] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Aug 30 01:03:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:49.034368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e560cf1-84ed-4fe5-aa7b-509e0788e0ce None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:49.082419 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e2a5c-c859-440a-8387-f98040cec755 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:49.132450 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:49.140078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fecf18-582b-4fff-8821-1d8934fb9418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:49.140700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fecf18-582b-4fff-8821-1d8934fb9418 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.140994 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fecf18-582b-4fff-8821-1d8934fb9418 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.141238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fecf18-582b-4fff-8821-1d8934fb9418 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.141899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fecf18-582b-4fff-8821-1d8934fb9418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:49.142203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fecf18-582b-4fff-8821-1d8934fb9418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:49.143629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fecf18-582b-4fff-8821-1d8934fb9418 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:49.157363 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 401/821] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:49.158211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fecf18-582b-4fff-8821-1d8934fb9418 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:49.202958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e560cf1-84ed-4fe5-aa7b-509e0788e0ce None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:49.259891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfef126-2737-457e-9bb1-50ef0963ed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:49.260888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfef126-2737-457e-9bb1-50ef0963ed27 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.261342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfef126-2737-457e-9bb1-50ef0963ed27 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.261796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfef126-2737-457e-9bb1-50ef0963ed27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.273533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfef126-2737-457e-9bb1-50ef0963ed27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:49.276034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfef126-2737-457e-9bb1-50ef0963ed27 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-2dfef126-2737-457e-9bb1-50ef0963ed27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:49.296019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:49.297020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:49.297906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:49.297906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:49.297906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:49.297906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:49.297906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:49.297906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:49.297906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:49.297906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:49.298328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 421/822] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:49.299313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfef126-2737-457e-9bb1-50ef0963ed27 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:49.334570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fecf18-582b-4fff-8821-1d8934fb9418 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:49.357378 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:49.383850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ba7e2-4c1c-438f-9d81-45892b77dcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:49.384565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ba7e2-4c1c-438f-9d81-45892b77dcc0 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.384894 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ba7e2-4c1c-438f-9d81-45892b77dcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.385158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ba7e2-4c1c-438f-9d81-45892b77dcc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.385790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ba7e2-4c1c-438f-9d81-45892b77dcc0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:49.387192 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ba7e2-4c1c-438f-9d81-45892b77dcc0 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:49.424163 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 402/823] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:03:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:49.424163 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ba7e2-4c1c-438f-9d81-45892b77dcc0 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:49.469807 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfef126-2737-457e-9bb1-50ef0963ed27 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:49.526084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:49.559699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f6e9d-b50d-4f92-b190-87062a679735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:49.561207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f6e9d-b50d-4f92-b190-87062a679735 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.561496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f6e9d-b50d-4f92-b190-87062a679735 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.561770 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f6e9d-b50d-4f92-b190-87062a679735 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.576153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f6e9d-b50d-4f92-b190-87062a679735 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:49.578964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f6e9d-b50d-4f92-b190-87062a679735 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-a67f6e9d-b50d-4f92-b190-87062a679735 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:49.601639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:49.602769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:49.603814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:49.603814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:49.603814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:49.603814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:49.603814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:49.603814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:49.603814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:49.603814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:49.603814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:03:49.603814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:49.608543 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 422/824] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Aug 30 01:03:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:49.611174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f6e9d-b50d-4f92-b190-87062a679735 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:49.636526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ba7e2-4c1c-438f-9d81-45892b77dcc0 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:49.684949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bd0c5-131c-4369-b2d3-16e1ee33437a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:49.686467 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bd0c5-131c-4369-b2d3-16e1ee33437a None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.686467 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bd0c5-131c-4369-b2d3-16e1ee33437a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.686689 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bd0c5-131c-4369-b2d3-16e1ee33437a None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.687431 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bd0c5-131c-4369-b2d3-16e1ee33437a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:49.687701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bd0c5-131c-4369-b2d3-16e1ee33437a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:49.690137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bd0c5-131c-4369-b2d3-16e1ee33437a None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:49.710640 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 403/825] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:49.715888 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bd0c5-131c-4369-b2d3-16e1ee33437a None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:49.775519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:49.796590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f6e9d-b50d-4f92-b190-87062a679735 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:49.847383 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bd0c5-131c-4369-b2d3-16e1ee33437a None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:49.856868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:49.858794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf44d3-a054-4111-bd18-f476a28eea5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:49.861416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf44d3-a054-4111-bd18-f476a28eea5e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.862869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf44d3-a054-4111-bd18-f476a28eea5e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.864559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf44d3-a054-4111-bd18-f476a28eea5e None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.866719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf44d3-a054-4111-bd18-f476a28eea5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:49.867565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf44d3-a054-4111-bd18-f476a28eea5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:49.870535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf44d3-a054-4111-bd18-f476a28eea5e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:49.900375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 423/826] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Aug 30 01:03:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:49.901858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf44d3-a054-4111-bd18-f476a28eea5e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:49.928349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5953ea9-f14c-4e6a-9764-abbd68df2a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:49.929694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5953ea9-f14c-4e6a-9764-abbd68df2a03 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.930187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5953ea9-f14c-4e6a-9764-abbd68df2a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.931113 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5953ea9-f14c-4e6a-9764-abbd68df2a03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.932810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5953ea9-f14c-4e6a-9764-abbd68df2a03 None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=049399a752ed4646a13f8e4a3a62d8e9) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:49.933371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5953ea9-f14c-4e6a-9764-abbd68df2a03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:49.937154 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5953ea9-f14c-4e6a-9764-abbd68df2a03 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:49.951641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 404/827] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:49 2022] GET /identity/v3/role_assignments?user.id=049399a752ed4646a13f8e4a3a62d8e9&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:50.068212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf44d3-a054-4111-bd18-f476a28eea5e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:50.077091 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5e8a5-8f83-47d7-a422-92ebe59e38b9 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.077091 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5e8a5-8f83-47d7-a422-92ebe59e38b9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.077492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5e8a5-8f83-47d7-a422-92ebe59e38b9 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.079061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 405/828] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:50.086523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528fccf-1537-4789-ad10-76d2a6fa5572 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.086523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528fccf-1537-4789-ad10-76d2a6fa5572 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.086900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528fccf-1537-4789-ad10-76d2a6fa5572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.108650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-a528fccf-1537-4789-ad10-76d2a6fa5572 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:50.110149 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a528fccf-1537-4789-ad10-76d2a6fa5572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:50.129429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2f1d8-cb5f-4844-8812-a2bf36ae9e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:50.130546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2f1d8-cb5f-4844-8812-a2bf36ae9e51 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.131091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2f1d8-cb5f-4844-8812-a2bf36ae9e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.131674 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2f1d8-cb5f-4844-8812-a2bf36ae9e51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.152359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2f1d8-cb5f-4844-8812-a2bf36ae9e51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:50.154251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2f1d8-cb5f-4844-8812-a2bf36ae9e51 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-78e2f1d8-cb5f-4844-8812-a2bf36ae9e51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:50.175592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:50.177088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:50.178226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:50.178226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:50.178226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:50.178226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 424/829] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:50.182673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2f1d8-cb5f-4844-8812-a2bf36ae9e51 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:50.191013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:50.239509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 406/830] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:03:50.369022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2f1d8-cb5f-4844-8812-a2bf36ae9e51 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:50.417468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e185eb1-46fe-4653-87e0-2876f185d2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:50.419048 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e185eb1-46fe-4653-87e0-2876f185d2fb None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.421101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e185eb1-46fe-4653-87e0-2876f185d2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.421974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e185eb1-46fe-4653-87e0-2876f185d2fb None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.423367 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e185eb1-46fe-4653-87e0-2876f185d2fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:50.424157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e185eb1-46fe-4653-87e0-2876f185d2fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:50.426953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e185eb1-46fe-4653-87e0-2876f185d2fb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:50.452529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 425/831] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:50.457652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528fccf-1537-4789-ad10-76d2a6fa5572 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:50.595155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e185eb1-46fe-4653-87e0-2876f185d2fb None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:50.621629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528fccf-1537-4789-ad10-76d2a6fa5572 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:50.640432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:50.667532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e423c-624f-45c7-b0c9-39e5108b3394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:50.669200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e423c-624f-45c7-b0c9-39e5108b3394 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.671094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e423c-624f-45c7-b0c9-39e5108b3394 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.671589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e423c-624f-45c7-b0c9-39e5108b3394 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.677853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:50.682037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e423c-624f-45c7-b0c9-39e5108b3394 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:50.691654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e423c-624f-45c7-b0c9-39e5108b3394 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-fd3e423c-624f-45c7-b0c9-39e5108b3394 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:50.723156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:50.724970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:50.726631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:50.726631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:50.726631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:50.726631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:50.726631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:50.726631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:50.726631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:50.726631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:50.727465 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e185eb1-46fe-4653-87e0-2876f185d2fb None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:50.735356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 407/832] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:50.741261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e423c-624f-45c7-b0c9-39e5108b3394 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:50.772987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbb03a-f35d-41c5-b117-b9572ff4b31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:50.774192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbb03a-f35d-41c5-b117-b9572ff4b31d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.774476 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbb03a-f35d-41c5-b117-b9572ff4b31d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.774752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbb03a-f35d-41c5-b117-b9572ff4b31d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.791501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbb03a-f35d-41c5-b117-b9572ff4b31d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-c4fbb03a-f35d-41c5-b117-b9572ff4b31d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:50.799450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:50.800502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:50.801358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:50.801358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:50.801358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:50.801358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:50.801358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:50.801358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:50.801358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:50.801358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:50.801358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:50.801358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:50.801358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:03:50.801358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:50.804077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 426/833] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:50.807345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbb03a-f35d-41c5-b117-b9572ff4b31d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:50.901298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e423c-624f-45c7-b0c9-39e5108b3394 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:50.953464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa6156-cff8-4593-9701-26e775f87d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:50.953744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa6156-cff8-4593-9701-26e775f87d40 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.953979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa6156-cff8-4593-9701-26e775f87d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.954195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa6156-cff8-4593-9701-26e775f87d40 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.954969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa6156-cff8-4593-9701-26e775f87d40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:50.955282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa6156-cff8-4593-9701-26e775f87d40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:50.956896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa6156-cff8-4593-9701-26e775f87d40 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:50.962631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbb03a-f35d-41c5-b117-b9572ff4b31d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:50.977725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 408/834] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:50.978740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa6156-cff8-4593-9701-26e775f87d40 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:50.995387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:51.010761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:51.013831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17125f84-3556-4b90-8986-572dc6038e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.014879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17125f84-3556-4b90-8986-572dc6038e23 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.015424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17125f84-3556-4b90-8986-572dc6038e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.015950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17125f84-3556-4b90-8986-572dc6038e23 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.016902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17125f84-3556-4b90-8986-572dc6038e23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:51.017475 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17125f84-3556-4b90-8986-572dc6038e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.019201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17125f84-3556-4b90-8986-572dc6038e23 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.034874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 427/835] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:03:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:51.036373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17125f84-3556-4b90-8986-572dc6038e23 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.156912 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa6156-cff8-4593-9701-26e775f87d40 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.181481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17125f84-3556-4b90-8986-572dc6038e23 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.216712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff4bc4-f784-4773-8dbe-3beaae3d490e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.217697 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff4bc4-f784-4773-8dbe-3beaae3d490e None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.218031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff4bc4-f784-4773-8dbe-3beaae3d490e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.218407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff4bc4-f784-4773-8dbe-3beaae3d490e None admin] PATH_INFO: `/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/8a3c3027b61d4f75a67e0a1bb8b52903/roles/d3cef78b3402418ea77f177cc74fecb1` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.237972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff4bc4-f784-4773-8dbe-3beaae3d490e None admin] RBAC: Authorizing `identity:create_grant(project_id=9188023bf51d4737b801e3fbd9af95b1, user_id=8a3c3027b61d4f75a67e0a1bb8b52903, role_id=d3cef78b3402418ea77f177cc74fecb1)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.239643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff4bc4-f784-4773-8dbe-3beaae3d490e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.241644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d98d0f-f644-4a7e-a918-156568a101ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.242938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d98d0f-f644-4a7e-a918-156568a101ef None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.243584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d98d0f-f644-4a7e-a918-156568a101ef None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.244119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d98d0f-f644-4a7e-a918-156568a101ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.263879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d98d0f-f644-4a7e-a918-156568a101ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.264362 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 409/836] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:03:50 2022] PUT /identity/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/8a3c3027b61d4f75a67e0a1bb8b52903/roles/d3cef78b3402418ea77f177cc74fecb1 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:03:51.266714 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d98d0f-f644-4a7e-a918-156568a101ef None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-41d98d0f-f644-4a7e-a918-156568a101ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:51.295473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:51.296756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:51.297699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:51.297699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:51.297699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:51.297699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 428/837] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:51.304858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff4bc4-f784-4773-8dbe-3beaae3d490e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.437285 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:51.503156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff4bc4-f784-4773-8dbe-3beaae3d490e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.565826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5421144-6893-4da3-8575-ab814e92eb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.566937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5421144-6893-4da3-8575-ab814e92eb4f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.567687 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5421144-6893-4da3-8575-ab814e92eb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.568426 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5421144-6893-4da3-8575-ab814e92eb4f None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.570190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5421144-6893-4da3-8575-ab814e92eb4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:51.571025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5421144-6893-4da3-8575-ab814e92eb4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.573391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5421144-6893-4da3-8575-ab814e92eb4f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.594417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 410/838] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:51.602862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d98d0f-f644-4a7e-a918-156568a101ef None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.645786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:51.718827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d98d0f-f644-4a7e-a918-156568a101ef None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.731772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdad030-0e3e-42bd-9efe-3e106959fe18 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.732109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdad030-0e3e-42bd-9efe-3e106959fe18 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.732512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdad030-0e3e-42bd-9efe-3e106959fe18 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.740171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 411/839] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:51.741997 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af8ee0-cce9-4e0c-907c-d4a3b0e569f8 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.742280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af8ee0-cce9-4e0c-907c-d4a3b0e569f8 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.742594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af8ee0-cce9-4e0c-907c-d4a3b0e569f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.757461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bba03a-2ac3-43c1-a5ff-e52b7e081dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.758275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bba03a-2ac3-43c1-a5ff-e52b7e081dbe None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.758761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bba03a-2ac3-43c1-a5ff-e52b7e081dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.759114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bba03a-2ac3-43c1-a5ff-e52b7e081dbe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.772824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bba03a-2ac3-43c1-a5ff-e52b7e081dbe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.774997 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bba03a-2ac3-43c1-a5ff-e52b7e081dbe None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:51.779627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-07af8ee0-cce9-4e0c-907c-d4a3b0e569f8 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:51.781923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-07af8ee0-cce9-4e0c-907c-d4a3b0e569f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-02bba03a-2ac3-43c1-a5ff-e52b7e081dbe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:51.794593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:51.796076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:51.797086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:51.797086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:51.797086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:51.797086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:51.797086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:51.797086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:51.797086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:51.797086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:51.797086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:03:51.797086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:51.797086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 429/840] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:03:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:51.809507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bba03a-2ac3-43c1-a5ff-e52b7e081dbe None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:51.890113 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 412/841] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:03:51.944386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:51.951633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bba03a-2ac3-43c1-a5ff-e52b7e081dbe None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:51.991396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360555c2-4300-4343-bfcf-aeeb684748b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:51.992023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360555c2-4300-4343-bfcf-aeeb684748b7 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.992319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360555c2-4300-4343-bfcf-aeeb684748b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.992592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360555c2-4300-4343-bfcf-aeeb684748b7 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.993328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360555c2-4300-4343-bfcf-aeeb684748b7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:51.993638 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360555c2-4300-4343-bfcf-aeeb684748b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:51.995147 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360555c2-4300-4343-bfcf-aeeb684748b7 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.016974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 430/842] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Aug 30 01:03:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:52.022554 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af8ee0-cce9-4e0c-907c-d4a3b0e569f8 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.092960 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:52.140756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af8ee0-cce9-4e0c-907c-d4a3b0e569f8 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.178655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7e71c-144f-4e2f-8df0-624e6907c3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.179277 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7e71c-144f-4e2f-8df0-624e6907c3ea None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.179554 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7e71c-144f-4e2f-8df0-624e6907c3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.179824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7e71c-144f-4e2f-8df0-624e6907c3ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.207144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360555c2-4300-4343-bfcf-aeeb684748b7 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.275666 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7e71c-144f-4e2f-8df0-624e6907c3ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.277172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7e71c-144f-4e2f-8df0-624e6907c3ea None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.347179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360555c2-4300-4343-bfcf-aeeb684748b7 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.347895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:52.396488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e4237-8ad5-4630-a4a4-47a369a34bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.397134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e4237-8ad5-4630-a4a4-47a369a34bcb None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.397429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e4237-8ad5-4630-a4a4-47a369a34bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.397687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e4237-8ad5-4630-a4a4-47a369a34bcb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.398283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e4237-8ad5-4630-a4a4-47a369a34bcb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.399659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e4237-8ad5-4630-a4a4-47a369a34bcb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.400862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 413/843] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:03:52.402827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e952b7-654f-465b-afaa-ccd856bd76ab None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.403318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e952b7-654f-465b-afaa-ccd856bd76ab None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.403773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e952b7-654f-465b-afaa-ccd856bd76ab None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.410797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 414/844] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:52.410797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e952b7-654f-465b-afaa-ccd856bd76ab None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-ff0e4237-8ad5-4630-a4a4-47a369a34bcb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:52.421080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:52.423098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:52.424199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 431/845] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:03:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:52.427076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c68be-7b78-4d9a-bc8a-637236b980be None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.427076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c68be-7b78-4d9a-bc8a-637236b980be None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.427076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c68be-7b78-4d9a-bc8a-637236b980be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.454298 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-072c68be-7b78-4d9a-bc8a-637236b980be None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:52.456360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-072c68be-7b78-4d9a-bc8a-637236b980be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:52.481060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:52.573639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 432/846] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:03:52.575193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c68be-7b78-4d9a-bc8a-637236b980be None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.587604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e952b7-654f-465b-afaa-ccd856bd76ab None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.649638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a9244-c75f-40b4-ac44-6eb32677be42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.650590 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a9244-c75f-40b4-ac44-6eb32677be42 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.651026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a9244-c75f-40b4-ac44-6eb32677be42 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.651493 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a9244-c75f-40b4-ac44-6eb32677be42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.665515 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a9244-c75f-40b4-ac44-6eb32677be42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.677855 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a9244-c75f-40b4-ac44-6eb32677be42 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-4c5a9244-c75f-40b4-ac44-6eb32677be42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:52.692555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:52.693785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:52.694839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:52.694839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:52.694839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:52.698910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 415/847] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:52.700021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a9244-c75f-40b4-ac44-6eb32677be42 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.724451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:52.750829 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c68be-7b78-4d9a-bc8a-637236b980be None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.797050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c18a2e-c082-432e-8669-1e5cd1000d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.797050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c18a2e-c082-432e-8669-1e5cd1000d00 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.797593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c18a2e-c082-432e-8669-1e5cd1000d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.797994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c18a2e-c082-432e-8669-1e5cd1000d00 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.798790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c18a2e-c082-432e-8669-1e5cd1000d00 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:52.799197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c18a2e-c082-432e-8669-1e5cd1000d00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.800864 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c18a2e-c082-432e-8669-1e5cd1000d00 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.812565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 433/848] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:03:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:52.814259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c18a2e-c082-432e-8669-1e5cd1000d00 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.832254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:52.839827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a9244-c75f-40b4-ac44-6eb32677be42 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.886438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f24baf-4e7c-417d-bfaa-8b481d7bb7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.887573 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f24baf-4e7c-417d-bfaa-8b481d7bb7bd None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.887573 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f24baf-4e7c-417d-bfaa-8b481d7bb7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.887882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f24baf-4e7c-417d-bfaa-8b481d7bb7bd None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.889189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f24baf-4e7c-417d-bfaa-8b481d7bb7bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:52.889496 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f24baf-4e7c-417d-bfaa-8b481d7bb7bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.891050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f24baf-4e7c-417d-bfaa-8b481d7bb7bd None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.908184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 416/849] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:52.909650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f24baf-4e7c-417d-bfaa-8b481d7bb7bd None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.946851 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c18a2e-c082-432e-8669-1e5cd1000d00 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.991192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58c743-d7f7-4c39-8204-75419a3b344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.991861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58c743-d7f7-4c39-8204-75419a3b344b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.992166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58c743-d7f7-4c39-8204-75419a3b344b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.992439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58c743-d7f7-4c39-8204-75419a3b344b None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.993089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58c743-d7f7-4c39-8204-75419a3b344b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:52.993425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58c743-d7f7-4c39-8204-75419a3b344b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.995048 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58c743-d7f7-4c39-8204-75419a3b344b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.001157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 434/850] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:03:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:53.002181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58c743-d7f7-4c39-8204-75419a3b344b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.033294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:53.070769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f24baf-4e7c-417d-bfaa-8b481d7bb7bd None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.125276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:53.126804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294499e-617b-4556-9e13-31270b412805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.129034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294499e-617b-4556-9e13-31270b412805 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.129597 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294499e-617b-4556-9e13-31270b412805 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.130076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294499e-617b-4556-9e13-31270b412805 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.131006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294499e-617b-4556-9e13-31270b412805 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.133046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294499e-617b-4556-9e13-31270b412805 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-8294499e-617b-4556-9e13-31270b412805 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:53.144759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:53.145760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:03:53.146721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:53.149734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 417/851] 10.222.0.44 () {56 vars in 1132 bytes} [Tue Aug 30 01:03:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:53.156337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294499e-617b-4556-9e13-31270b412805 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.164087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58c743-d7f7-4c39-8204-75419a3b344b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.212387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c0759-0a52-41d9-b2e2-fe4124c687fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.214104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c0759-0a52-41d9-b2e2-fe4124c687fa None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.214435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c0759-0a52-41d9-b2e2-fe4124c687fa None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.214710 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c0759-0a52-41d9-b2e2-fe4124c687fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.225517 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c0759-0a52-41d9-b2e2-fe4124c687fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.227011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c0759-0a52-41d9-b2e2-fe4124c687fa None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-bd4c0759-0a52-41d9-b2e2-fe4124c687fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:53.241907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:53.242996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:53.243981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:53.243981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:53.243981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:53.243981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:53.243981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:53.243981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:53.243981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:53.243981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:53.243981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 435/852] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:53.251425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c0759-0a52-41d9-b2e2-fe4124c687fa None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.296667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294499e-617b-4556-9e13-31270b412805 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.336992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f10c5-24bc-4944-96af-b908054ae110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.337614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f10c5-24bc-4944-96af-b908054ae110 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.337912 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f10c5-24bc-4944-96af-b908054ae110 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.338227 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f10c5-24bc-4944-96af-b908054ae110 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.338873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f10c5-24bc-4944-96af-b908054ae110 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:53.339177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f10c5-24bc-4944-96af-b908054ae110 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.340585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f10c5-24bc-4944-96af-b908054ae110 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.343006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:53.353238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 418/853] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Aug 30 01:03:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:53.354208 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f10c5-24bc-4944-96af-b908054ae110 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.370359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c0759-0a52-41d9-b2e2-fe4124c687fa None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.408343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae335f7f-5b00-43cf-9472-cc1ac7f2f72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.409093 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335f7f-5b00-43cf-9472-cc1ac7f2f72c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.409484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335f7f-5b00-43cf-9472-cc1ac7f2f72c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.409825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335f7f-5b00-43cf-9472-cc1ac7f2f72c None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.410652 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae335f7f-5b00-43cf-9472-cc1ac7f2f72c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:53.411025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae335f7f-5b00-43cf-9472-cc1ac7f2f72c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.412672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae335f7f-5b00-43cf-9472-cc1ac7f2f72c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.415667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:53.428132 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 436/854] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:53.435340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae335f7f-5b00-43cf-9472-cc1ac7f2f72c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.481217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f10c5-24bc-4944-96af-b908054ae110 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.521390 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6831b2b-f67d-49ce-a924-0a1d5a37fc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.522133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6831b2b-f67d-49ce-a924-0a1d5a37fc58 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.522914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6831b2b-f67d-49ce-a924-0a1d5a37fc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.523303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6831b2b-f67d-49ce-a924-0a1d5a37fc58 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.524176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6831b2b-f67d-49ce-a924-0a1d5a37fc58 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:53.524527 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6831b2b-f67d-49ce-a924-0a1d5a37fc58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.526466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6831b2b-f67d-49ce-a924-0a1d5a37fc58 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.540074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 419/855] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Aug 30 01:03:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:53.540074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643daf16-7e25-4995-9d78-b978aacad62b None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.540074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643daf16-7e25-4995-9d78-b978aacad62b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.540074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643daf16-7e25-4995-9d78-b978aacad62b None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.540074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 420/856] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:53.550132 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87dd6a-11db-4252-81c7-6452b23b52fa None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.550560 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87dd6a-11db-4252-81c7-6452b23b52fa None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.550845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87dd6a-11db-4252-81c7-6452b23b52fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.564466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae335f7f-5b00-43cf-9472-cc1ac7f2f72c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.577305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-7b87dd6a-11db-4252-81c7-6452b23b52fa None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:53.579361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7b87dd6a-11db-4252-81c7-6452b23b52fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:53.610814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a295149-35a5-46df-82a0-21c30cc5c3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.611802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a295149-35a5-46df-82a0-21c30cc5c3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.612195 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a295149-35a5-46df-82a0-21c30cc5c3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.612531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a295149-35a5-46df-82a0-21c30cc5c3a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.613376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a295149-35a5-46df-82a0-21c30cc5c3a1 None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=3330df65b64c4fffbc3bb687d5c51021) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:53.613748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a295149-35a5-46df-82a0-21c30cc5c3a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.615344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a295149-35a5-46df-82a0-21c30cc5c3a1 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.628019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 437/857] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:53 2022] GET /identity/v3/role_assignments?user.id=3330df65b64c4fffbc3bb687d5c51021&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:53.662375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:53.680164 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 421/858] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:03:53.951703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a295149-35a5-46df-82a0-21c30cc5c3a1 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:54.018174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:54.066383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a295149-35a5-46df-82a0-21c30cc5c3a1 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:54.103647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206007e4-800d-458c-9a04-d4750ba8c1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:54.104594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206007e4-800d-458c-9a04-d4750ba8c1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:54.105036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206007e4-800d-458c-9a04-d4750ba8c1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:54.105462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206007e4-800d-458c-9a04-d4750ba8c1c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:54.117191 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206007e4-800d-458c-9a04-d4750ba8c1c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-206007e4-800d-458c-9a04-d4750ba8c1c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:54.123440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:54.124748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:54.125926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:54.125926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:54.125926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:54.125926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:54.125926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:54.125926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:54.125926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:54.125926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:54.125926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:54.125926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:54.125926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:03:54.125926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:54.128478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 438/859] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:03:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:54.137376 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87dd6a-11db-4252-81c7-6452b23b52fa None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:54.250001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87dd6a-11db-4252-81c7-6452b23b52fa None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:54.286434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47036bf0-cc37-4520-b1d3-dbdb5447eaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:54.287299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47036bf0-cc37-4520-b1d3-dbdb5447eaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:54.287717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47036bf0-cc37-4520-b1d3-dbdb5447eaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:54.288183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47036bf0-cc37-4520-b1d3-dbdb5447eaa5 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:54.289011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47036bf0-cc37-4520-b1d3-dbdb5447eaa5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:54.289463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47036bf0-cc37-4520-b1d3-dbdb5447eaa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:54.291324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47036bf0-cc37-4520-b1d3-dbdb5447eaa5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:54.302683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 422/860] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:03:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:54.308366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206007e4-800d-458c-9a04-d4750ba8c1c3 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:54.421777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206007e4-800d-458c-9a04-d4750ba8c1c3 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:54.444249 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:54.462250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7503c69-27b9-49ff-a484-9707cfa33cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:54.463153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7503c69-27b9-49ff-a484-9707cfa33cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:54.463635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7503c69-27b9-49ff-a484-9707cfa33cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:54.464102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7503c69-27b9-49ff-a484-9707cfa33cd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:54.475368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7503c69-27b9-49ff-a484-9707cfa33cd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:54.477191 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7503c69-27b9-49ff-a484-9707cfa33cd5 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-f7503c69-27b9-49ff-a484-9707cfa33cd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:54.492846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:54.493907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:54.494934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:54.494934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:54.494934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:54.496872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 439/861] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:54.505510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47036bf0-cc37-4520-b1d3-dbdb5447eaa5 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:54.578402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:54.628953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47036bf0-cc37-4520-b1d3-dbdb5447eaa5 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:54.665658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238ed6c-6e1f-4308-bcb5-52377d34b29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:54.666286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238ed6c-6e1f-4308-bcb5-52377d34b29c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:54.666556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238ed6c-6e1f-4308-bcb5-52377d34b29c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:54.666796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238ed6c-6e1f-4308-bcb5-52377d34b29c None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:54.667443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238ed6c-6e1f-4308-bcb5-52377d34b29c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:54.667745 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238ed6c-6e1f-4308-bcb5-52377d34b29c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:54.670587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238ed6c-6e1f-4308-bcb5-52377d34b29c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:54.683580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 423/862] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:54.692451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7503c69-27b9-49ff-a484-9707cfa33cd5 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:54.812943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7503c69-27b9-49ff-a484-9707cfa33cd5 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:54.850175 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feac5f8-6084-4286-83cb-4cdb6614b8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:54.850809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feac5f8-6084-4286-83cb-4cdb6614b8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:54.851086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feac5f8-6084-4286-83cb-4cdb6614b8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:54.851357 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feac5f8-6084-4286-83cb-4cdb6614b8a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:54.861559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feac5f8-6084-4286-83cb-4cdb6614b8a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:54.863071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feac5f8-6084-4286-83cb-4cdb6614b8a3 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-3feac5f8-6084-4286-83cb-4cdb6614b8a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:03:54.877618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:54.878627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:54.880051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:54.880051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:54.880051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:03:54.880051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:03:54.880051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:03:54.880051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:03:54.880051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:03:54.880051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:03:54.880051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:03:54.880051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:54.880051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 440/863] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Aug 30 01:03:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:54.886665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238ed6c-6e1f-4308-bcb5-52377d34b29c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:54.995781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238ed6c-6e1f-4308-bcb5-52377d34b29c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:55.014685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:55.034364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e682ad-2258-475c-829e-4dcb58089deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:55.034971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e682ad-2258-475c-829e-4dcb58089deb None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.035246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e682ad-2258-475c-829e-4dcb58089deb None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.035514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e682ad-2258-475c-829e-4dcb58089deb None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.036273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e682ad-2258-475c-829e-4dcb58089deb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:55.036576 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e682ad-2258-475c-829e-4dcb58089deb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:55.037999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e682ad-2258-475c-829e-4dcb58089deb None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:55.040291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192765c6-4396-4481-b82d-7dd30dbaebd0 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.040736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192765c6-4396-4481-b82d-7dd30dbaebd0 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.040837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192765c6-4396-4481-b82d-7dd30dbaebd0 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.042430 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 441/864] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:55.048471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d076f53-74e8-41dc-bdcb-a70095a87fc3 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.048755 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d076f53-74e8-41dc-bdcb-a70095a87fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.049029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d076f53-74e8-41dc-bdcb-a70095a87fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.059716 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 424/865] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Aug 30 01:03:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:55.071358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e682ad-2258-475c-829e-4dcb58089deb None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:55.074947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-3d076f53-74e8-41dc-bdcb-a70095a87fc3 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:55.076427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3d076f53-74e8-41dc-bdcb-a70095a87fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:55.113346 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:55.172741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 442/866] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:03:55.182985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e682ad-2258-475c-829e-4dcb58089deb None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:55.225065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33414ca0-6efd-45b9-8494-2437d7c21719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:55.225762 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33414ca0-6efd-45b9-8494-2437d7c21719 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.226042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33414ca0-6efd-45b9-8494-2437d7c21719 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.226314 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33414ca0-6efd-45b9-8494-2437d7c21719 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.275865 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:55.305341 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33414ca0-6efd-45b9-8494-2437d7c21719 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:55.306892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33414ca0-6efd-45b9-8494-2437d7c21719 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:55.420270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 425/867] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:03:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:03:55.421753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d076f53-74e8-41dc-bdcb-a70095a87fc3 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:55.456022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33414ca0-6efd-45b9-8494-2437d7c21719 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:55.552894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d076f53-74e8-41dc-bdcb-a70095a87fc3 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:55.556454 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:55.602527 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33414ca0-6efd-45b9-8494-2437d7c21719 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:55.610084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3273f-203e-452d-988a-9bdee05ed99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:55.611041 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3273f-203e-452d-988a-9bdee05ed99c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.611435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3273f-203e-452d-988a-9bdee05ed99c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.611818 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3273f-203e-452d-988a-9bdee05ed99c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.627213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3273f-203e-452d-988a-9bdee05ed99c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:55.629213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3273f-203e-452d-988a-9bdee05ed99c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-f6d3273f-203e-452d-988a-9bdee05ed99c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:03:55.663639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:55.665271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:03:55.666865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:03:55.666865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:03:55.666865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:55.671310 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 443/868] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:55.675113 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff1aa0-d222-432e-b63f-60f70e43e0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:55.676050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3273f-203e-452d-988a-9bdee05ed99c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:55.677179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff1aa0-d222-432e-b63f-60f70e43e0b3 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.677868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff1aa0-d222-432e-b63f-60f70e43e0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.678535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff1aa0-d222-432e-b63f-60f70e43e0b3 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.679880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff1aa0-d222-432e-b63f-60f70e43e0b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:55.684445 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff1aa0-d222-432e-b63f-60f70e43e0b3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:55.716988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 426/869] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:03:55 2022] POST /identity/v3/services => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:55.719527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:55.745743 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee73e4-cfed-49e6-aafb-408c878d1aa1 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.746060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee73e4-cfed-49e6-aafb-408c878d1aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.746376 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee73e4-cfed-49e6-aafb-408c878d1aa1 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.748105 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 427/870] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:55.753661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea23eb-80f2-41b7-be32-d95fc4764625 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.753959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea23eb-80f2-41b7-be32-d95fc4764625 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.754215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea23eb-80f2-41b7-be32-d95fc4764625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.775489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-b9ea23eb-80f2-41b7-be32-d95fc4764625 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:55.777033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b9ea23eb-80f2-41b7-be32-d95fc4764625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:55.865200 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3273f-203e-452d-988a-9bdee05ed99c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:55.870608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 428/871] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:03:55.872804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f031a-06b1-4877-a381-853a94ea219b None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.873143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f031a-06b1-4877-a381-853a94ea219b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.873536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f031a-06b1-4877-a381-853a94ea219b None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.875020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 429/872] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:55.882009 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b2d7a-8498-461e-87ee-6b49bb6c57e8 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.882335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b2d7a-8498-461e-87ee-6b49bb6c57e8 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.882602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b2d7a-8498-461e-87ee-6b49bb6c57e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.905083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-630b2d7a-8498-461e-87ee-6b49bb6c57e8 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:55.906332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-630b2d7a-8498-461e-87ee-6b49bb6c57e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:55.920992 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d5375-8368-4fc5-a5e9-0391edd24ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:55.922055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d5375-8368-4fc5-a5e9-0391edd24ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:55.922194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d5375-8368-4fc5-a5e9-0391edd24ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:55.922433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d5375-8368-4fc5-a5e9-0391edd24ab6 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:55.923154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d5375-8368-4fc5-a5e9-0391edd24ab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:55.923573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d5375-8368-4fc5-a5e9-0391edd24ab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:55.924925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d5375-8368-4fc5-a5e9-0391edd24ab6 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:55.944054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 444/873] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:55.947626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d5375-8368-4fc5-a5e9-0391edd24ab6 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:55.982147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:55.997989 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 430/874] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:03:56.042281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:56.070451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d5375-8368-4fc5-a5e9-0391edd24ab6 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:56.106703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349cb3b-3908-4005-806c-a73858d1b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:56.107371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349cb3b-3908-4005-806c-a73858d1b25a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.107647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349cb3b-3908-4005-806c-a73858d1b25a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.107968 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349cb3b-3908-4005-806c-a73858d1b25a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.115041 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349cb3b-3908-4005-806c-a73858d1b25a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:56.116593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349cb3b-3908-4005-806c-a73858d1b25a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-0349cb3b-3908-4005-806c-a73858d1b25a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:03:56.129027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:56.129950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:56.130927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:03:56.130927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:03:56.130927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:03:56.130927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:03:56.130927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:03:56.130927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:03:56.130927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:03:56.130927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:56.134936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 445/875] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:03:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:56.136392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b2d7a-8498-461e-87ee-6b49bb6c57e8 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:56.160052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349cb3b-3908-4005-806c-a73858d1b25a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:56.245450 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b2d7a-8498-461e-87ee-6b49bb6c57e8 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:56.284286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349cb3b-3908-4005-806c-a73858d1b25a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:56.284848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617e9d9-2391-42ac-9e13-c608dd2f759c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:56.285467 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617e9d9-2391-42ac-9e13-c608dd2f759c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.285742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617e9d9-2391-42ac-9e13-c608dd2f759c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.286010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617e9d9-2391-42ac-9e13-c608dd2f759c None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.286704 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617e9d9-2391-42ac-9e13-c608dd2f759c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:56.287060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617e9d9-2391-42ac-9e13-c608dd2f759c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:56.288617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617e9d9-2391-42ac-9e13-c608dd2f759c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:56.302321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 431/876] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:03:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:56.307487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617e9d9-2391-42ac-9e13-c608dd2f759c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:56.329246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e19f01-0f2a-4a2a-a716-0cc83d13fc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:56.330186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e19f01-0f2a-4a2a-a716-0cc83d13fc42 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.330468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e19f01-0f2a-4a2a-a716-0cc83d13fc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.330785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e19f01-0f2a-4a2a-a716-0cc83d13fc42 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.331408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e19f01-0f2a-4a2a-a716-0cc83d13fc42 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:56.333283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e19f01-0f2a-4a2a-a716-0cc83d13fc42 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:56.336815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:56.350834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 446/877] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:03:56 2022] POST /identity/v3/services => generated 269 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:56.352062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e19f01-0f2a-4a2a-a716-0cc83d13fc42 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:56.372398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:56.448982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617e9d9-2391-42ac-9e13-c608dd2f759c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:56.499105 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e19f01-0f2a-4a2a-a716-0cc83d13fc42 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:56.503647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1d1cc-121f-4413-90a7-346fcb1e150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:56.505022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1d1cc-121f-4413-90a7-346fcb1e150e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.505600 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1d1cc-121f-4413-90a7-346fcb1e150e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.506108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1d1cc-121f-4413-90a7-346fcb1e150e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.506972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1d1cc-121f-4413-90a7-346fcb1e150e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:56.508959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1d1cc-121f-4413-90a7-346fcb1e150e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-2fd1d1cc-121f-4413-90a7-346fcb1e150e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:56.522255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:56.523905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:03:56.525512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:56.528291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 432/878] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Aug 30 01:03:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:56.530730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1d1cc-121f-4413-90a7-346fcb1e150e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:56.558223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d42d9-8913-49e8-9115-f483078b7ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:56.559020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d42d9-8913-49e8-9115-f483078b7ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.559296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d42d9-8913-49e8-9115-f483078b7ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.560295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d42d9-8913-49e8-9115-f483078b7ed0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.560995 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d42d9-8913-49e8-9115-f483078b7ed0 None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=8a3c3027b61d4f75a67e0a1bb8b52903) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:56.561186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d42d9-8913-49e8-9115-f483078b7ed0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:56.562696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d42d9-8913-49e8-9115-f483078b7ed0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:56.573602 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 447/879] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:03:56 2022] GET /identity/v3/role_assignments?user.id=8a3c3027b61d4f75a67e0a1bb8b52903&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:56.685729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1d1cc-121f-4413-90a7-346fcb1e150e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:56.706436 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:56.741207 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cee491-f5c8-4a47-a209-6a477cc6c556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:56.742282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cee491-f5c8-4a47-a209-6a477cc6c556 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.742790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cee491-f5c8-4a47-a209-6a477cc6c556 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.743269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cee491-f5c8-4a47-a209-6a477cc6c556 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.744233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cee491-f5c8-4a47-a209-6a477cc6c556 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:56.744779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cee491-f5c8-4a47-a209-6a477cc6c556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:56.746522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cee491-f5c8-4a47-a209-6a477cc6c556 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:56.753884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 433/880] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Aug 30 01:03:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:56.760972 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d42d9-8913-49e8-9115-f483078b7ed0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:56.827617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:56.886325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d42d9-8913-49e8-9115-f483078b7ed0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:56.926482 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3e4d0-1be5-4690-98cd-d4f254c8f313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:56.927179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3e4d0-1be5-4690-98cd-d4f254c8f313 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.927496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3e4d0-1be5-4690-98cd-d4f254c8f313 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.927769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3e4d0-1be5-4690-98cd-d4f254c8f313 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.928513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3e4d0-1be5-4690-98cd-d4f254c8f313 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:56.928847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3e4d0-1be5-4690-98cd-d4f254c8f313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:56.930363 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3e4d0-1be5-4690-98cd-d4f254c8f313 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:56.937044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 448/881] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Aug 30 01:03:56 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:57.892504 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4259457-f166-4f4f-b6ad-32bea2873e7c None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:57.893492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4259457-f166-4f4f-b6ad-32bea2873e7c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:57.893970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4259457-f166-4f4f-b6ad-32bea2873e7c None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:57.895786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 434/882] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:57.901295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec5d7a-841e-47db-ad60-e463c6485d2c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:57.902119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec5d7a-841e-47db-ad60-e463c6485d2c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:57.902756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec5d7a-841e-47db-ad60-e463c6485d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:57.922841 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d1ec5d7a-841e-47db-ad60-e463c6485d2c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:57.924638 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d1ec5d7a-841e-47db-ad60-e463c6485d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:58.003326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 449/883] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:03:58.297463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4259457-f166-4f4f-b6ad-32bea2873e7c None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.411766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4259457-f166-4f4f-b6ad-32bea2873e7c None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:58.451813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c12a7-72b4-4b2b-a7c4-e8f8f5013354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:58.452919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c12a7-72b4-4b2b-a7c4-e8f8f5013354 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.453444 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c12a7-72b4-4b2b-a7c4-e8f8f5013354 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.453909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c12a7-72b4-4b2b-a7c4-e8f8f5013354 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.454761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c12a7-72b4-4b2b-a7c4-e8f8f5013354 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:58.456559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c12a7-72b4-4b2b-a7c4-e8f8f5013354 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-3c8c12a7-72b4-4b2b-a7c4-e8f8f5013354 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:58.467478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:58.468558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:03:58.469381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:03:58.472158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 435/884] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:03:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:58.478263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec5d7a-841e-47db-ad60-e463c6485d2c None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.503931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4610f-ed9b-490c-b93c-df47001ea5db None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.504689 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4610f-ed9b-490c-b93c-df47001ea5db None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.504963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4610f-ed9b-490c-b93c-df47001ea5db None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.506581 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 436/885] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:03:58.511782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9bc3e-8a98-43a2-a6c5-ca8fd236c964 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.512037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9bc3e-8a98-43a2-a6c5-ca8fd236c964 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.512317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9bc3e-8a98-43a2-a6c5-ca8fd236c964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.538928 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:58.545808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-7dd9bc3e-8a98-43a2-a6c5-ca8fd236c964 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:58.547096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7dd9bc3e-8a98-43a2-a6c5-ca8fd236c964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:58.592500 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:58.612348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec5d7a-841e-47db-ad60-e463c6485d2c None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:58.636040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 437/886] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:03:58.660650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba28cec-67b0-4765-b6d4-b7370b55e198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:58.661268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba28cec-67b0-4765-b6d4-b7370b55e198 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.661543 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba28cec-67b0-4765-b6d4-b7370b55e198 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.661814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba28cec-67b0-4765-b6d4-b7370b55e198 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.662505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba28cec-67b0-4765-b6d4-b7370b55e198 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:58.662811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba28cec-67b0-4765-b6d4-b7370b55e198 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:58.664258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba28cec-67b0-4765-b6d4-b7370b55e198 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:58.670220 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 450/887] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:03:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:58.675521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9bc3e-8a98-43a2-a6c5-ca8fd236c964 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.780994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425c890-2b30-44ce-a13c-b62685ef2322 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.780994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425c890-2b30-44ce-a13c-b62685ef2322 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.780994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425c890-2b30-44ce-a13c-b62685ef2322 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.780994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 451/888] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:03:58.785177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9863-18a3-4f35-81fd-93eb8cad2c6d None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.785462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9863-18a3-4f35-81fd-93eb8cad2c6d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.785738 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9863-18a3-4f35-81fd-93eb8cad2c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.800736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9bc3e-8a98-43a2-a6c5-ca8fd236c964 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:58.806060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-f7ce9863-18a3-4f35-81fd-93eb8cad2c6d None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:58.807328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f7ce9863-18a3-4f35-81fd-93eb8cad2c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:58.844753 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb64137-bd1a-46a6-abc2-94e431d1b718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:58.845291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb64137-bd1a-46a6-abc2-94e431d1b718 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:58.845566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb64137-bd1a-46a6-abc2-94e431d1b718 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:58.845834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb64137-bd1a-46a6-abc2-94e431d1b718 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:58.846471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb64137-bd1a-46a6-abc2-94e431d1b718 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:58.846767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb64137-bd1a-46a6-abc2-94e431d1b718 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:58.848156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb64137-bd1a-46a6-abc2-94e431d1b718 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:58.854797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 438/889] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:03:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:58.861089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb64137-bd1a-46a6-abc2-94e431d1b718 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.895821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 452/890] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:03:58.904530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:58.947651 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9863-18a3-4f35-81fd-93eb8cad2c6d None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:58.976572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb64137-bd1a-46a6-abc2-94e431d1b718 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.004397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:59.015381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fcd33-eba8-4dc3-87c4-10c691781516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.016567 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fcd33-eba8-4dc3-87c4-10c691781516 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.016850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fcd33-eba8-4dc3-87c4-10c691781516 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.017359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fcd33-eba8-4dc3-87c4-10c691781516 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.017986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fcd33-eba8-4dc3-87c4-10c691781516 None admin] RBAC: Adding query filter params (interface=public, service_id=6e6619d316d44e0d945728685b002b53, region_id=RegionOne) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:59.018294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fcd33-eba8-4dc3-87c4-10c691781516 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:59.019767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fcd33-eba8-4dc3-87c4-10c691781516 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:59.025463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 439/891] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Aug 30 01:03:58 2022] GET /identity/v3/endpoints?service_id=6e6619d316d44e0d945728685b002b53&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:03:59.026603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a35ed7-4295-4b30-9513-d32f322f6db2 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.026878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a35ed7-4295-4b30-9513-d32f322f6db2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.027150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a35ed7-4295-4b30-9513-d32f322f6db2 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.032092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 440/892] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:59.032092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a35ed7-4295-4b30-9513-d32f322f6db2 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.078219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9863-18a3-4f35-81fd-93eb8cad2c6d None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.118454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e92db-6724-4f6d-9f38-2e4e70b86c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.119120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e92db-6724-4f6d-9f38-2e4e70b86c94 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.119389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e92db-6724-4f6d-9f38-2e4e70b86c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.119655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e92db-6724-4f6d-9f38-2e4e70b86c94 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.120265 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e92db-6724-4f6d-9f38-2e4e70b86c94 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:59.122440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e92db-6724-4f6d-9f38-2e4e70b86c94 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-d92e92db-6724-4f6d-9f38-2e4e70b86c94 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:59.132870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:59.133884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:59.134778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 453/893] 10.222.0.44 () {56 vars in 1132 bytes} [Tue Aug 30 01:03:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:03:59.136164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107781d5-e9fc-4b15-b9f7-60953f59467a None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.136164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107781d5-e9fc-4b15-b9f7-60953f59467a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.136164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107781d5-e9fc-4b15-b9f7-60953f59467a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.140842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:59.145354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a35ed7-4295-4b30-9513-d32f322f6db2 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.156961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-107781d5-e9fc-4b15-b9f7-60953f59467a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:59.158166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-107781d5-e9fc-4b15-b9f7-60953f59467a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:59.182506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb07ce0-5f3e-44ae-8292-2ec0e7cd0434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.183110 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb07ce0-5f3e-44ae-8292-2ec0e7cd0434 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.183388 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb07ce0-5f3e-44ae-8292-2ec0e7cd0434 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.183657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb07ce0-5f3e-44ae-8292-2ec0e7cd0434 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.184899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb07ce0-5f3e-44ae-8292-2ec0e7cd0434 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:59.186363 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb07ce0-5f3e-44ae-8292-2ec0e7cd0434 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:59.192906 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 441/894] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Aug 30 01:03:59 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:03:59.193842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb07ce0-5f3e-44ae-8292-2ec0e7cd0434 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.239141 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 454/895] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:03:59.240059 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107781d5-e9fc-4b15-b9f7-60953f59467a None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.297169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:59.318029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb07ce0-5f3e-44ae-8292-2ec0e7cd0434 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.360895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa2e9d-dd78-461e-8d36-84e4f65b9116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.361512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa2e9d-dd78-461e-8d36-84e4f65b9116 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.361789 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa2e9d-dd78-461e-8d36-84e4f65b9116 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.362060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa2e9d-dd78-461e-8d36-84e4f65b9116 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.362698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa2e9d-dd78-461e-8d36-84e4f65b9116 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:59.363001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa2e9d-dd78-461e-8d36-84e4f65b9116 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:59.364964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa2e9d-dd78-461e-8d36-84e4f65b9116 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:59.370059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 442/896] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Aug 30 01:03:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:59.377298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa2e9d-dd78-461e-8d36-84e4f65b9116 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.388155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107781d5-e9fc-4b15-b9f7-60953f59467a None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.414498 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:59.433000 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd428995-a13e-481f-8cd1-a6a49a4bfc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.433951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd428995-a13e-481f-8cd1-a6a49a4bfc33 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.435179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd428995-a13e-481f-8cd1-a6a49a4bfc33 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.435832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd428995-a13e-481f-8cd1-a6a49a4bfc33 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.448031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd428995-a13e-481f-8cd1-a6a49a4bfc33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-dd428995-a13e-481f-8cd1-a6a49a4bfc33 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:03:59.453903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:03:59.454830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:03:59.455692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:03:59.455692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:03:59.455692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:03:59.455692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:03:59.455692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:03:59.455692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:03:59.455692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:03:59.455692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:03:59.455692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:03:59.455692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:03:59.455692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:03:59.455692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:03:59.458192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 455/897] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Aug 30 01:03:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:03:59.465212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd428995-a13e-481f-8cd1-a6a49a4bfc33 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.494720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa2e9d-dd78-461e-8d36-84e4f65b9116 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.534625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d640ddc-6d56-452f-8747-673901611c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.535539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d640ddc-6d56-452f-8747-673901611c93 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.535818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d640ddc-6d56-452f-8747-673901611c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.536145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d640ddc-6d56-452f-8747-673901611c93 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.536904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d640ddc-6d56-452f-8747-673901611c93 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:59.537242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d640ddc-6d56-452f-8747-673901611c93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:59.538597 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d640ddc-6d56-452f-8747-673901611c93 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:59.547139 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 443/898] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Aug 30 01:03:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:59.547957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d640ddc-6d56-452f-8747-673901611c93 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.586632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:59.597558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd428995-a13e-481f-8cd1-a6a49a4bfc33 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.641264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0229a-96c6-42f6-a3e0-70d1060db19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.642002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0229a-96c6-42f6-a3e0-70d1060db19e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.642277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0229a-96c6-42f6-a3e0-70d1060db19e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.642544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0229a-96c6-42f6-a3e0-70d1060db19e None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.643840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0229a-96c6-42f6-a3e0-70d1060db19e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:59.644181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0229a-96c6-42f6-a3e0-70d1060db19e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:59.645582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0229a-96c6-42f6-a3e0-70d1060db19e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:59.655852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 456/899] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Aug 30 01:03:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:59.657128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0229a-96c6-42f6-a3e0-70d1060db19e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.671758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:59.681895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d640ddc-6d56-452f-8747-673901611c93 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.727092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017bb6b-0cba-4607-af1c-500129cced21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.727703 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017bb6b-0cba-4607-af1c-500129cced21 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.727979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017bb6b-0cba-4607-af1c-500129cced21 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.728276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017bb6b-0cba-4607-af1c-500129cced21 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.728931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017bb6b-0cba-4607-af1c-500129cced21 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:59.730298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017bb6b-0cba-4607-af1c-500129cced21 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:59.756221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 444/900] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:03:59 2022] POST /identity/v3/services => generated 241 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:03:59.757211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017bb6b-0cba-4607-af1c-500129cced21 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.786210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0229a-96c6-42f6-a3e0-70d1060db19e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.847303 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e464551-3c9d-45f8-891b-ffa2c754f02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.848283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e464551-3c9d-45f8-891b-ffa2c754f02a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.848571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e464551-3c9d-45f8-891b-ffa2c754f02a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.848843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e464551-3c9d-45f8-891b-ffa2c754f02a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.850104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e464551-3c9d-45f8-891b-ffa2c754f02a None admin] RBAC: Adding query filter params (interface=public, service_id=9c8e0777935a44569fb3b2331ec7c430, region_id=RegionOne) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:03:59.850425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e464551-3c9d-45f8-891b-ffa2c754f02a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:59.852199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e464551-3c9d-45f8-891b-ffa2c754f02a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:59.864336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 457/901] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Aug 30 01:03:59 2022] GET /identity/v3/endpoints?service_id=9c8e0777935a44569fb3b2331ec7c430&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:03:59.868200 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e464551-3c9d-45f8-891b-ffa2c754f02a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.890691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:59.909423 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017bb6b-0cba-4607-af1c-500129cced21 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:59.968692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe44c02-baeb-47a5-a378-be7c6bdfd67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:59.969340 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe44c02-baeb-47a5-a378-be7c6bdfd67e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.969622 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe44c02-baeb-47a5-a378-be7c6bdfd67e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.969897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe44c02-baeb-47a5-a378-be7c6bdfd67e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.992187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe44c02-baeb-47a5-a378-be7c6bdfd67e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:59.993758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe44c02-baeb-47a5-a378-be7c6bdfd67e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.002273 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e464551-3c9d-45f8-891b-ffa2c754f02a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-abe44c02-baeb-47a5-a378-be7c6bdfd67e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:00.009680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:00.010668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:00.011551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:00.011551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:00.011551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:00.011551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 445/902] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:00.017580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe44c02-baeb-47a5-a378-be7c6bdfd67e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:00.021654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:00.044954 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff468cf-51c7-4417-a74a-2a326d58c869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.045949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff468cf-51c7-4417-a74a-2a326d58c869 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.046432 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff468cf-51c7-4417-a74a-2a326d58c869 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.046902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff468cf-51c7-4417-a74a-2a326d58c869 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.047763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff468cf-51c7-4417-a74a-2a326d58c869 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.049442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff468cf-51c7-4417-a74a-2a326d58c869 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.056553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 458/903] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Aug 30 01:03:59 2022] GET /identity/v3/services => generated 1497 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:04:00.154056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe44c02-baeb-47a5-a378-be7c6bdfd67e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.193608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240adf9-e099-4315-b710-0cef604c1fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.194234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240adf9-e099-4315-b710-0cef604c1fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.194515 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240adf9-e099-4315-b710-0cef604c1fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.194789 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240adf9-e099-4315-b710-0cef604c1fc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.195449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240adf9-e099-4315-b710-0cef604c1fc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:00.195752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240adf9-e099-4315-b710-0cef604c1fc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.197196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240adf9-e099-4315-b710-0cef604c1fc0 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.211213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 446/904] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:00.217310 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff468cf-51c7-4417-a74a-2a326d58c869 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:00.319244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:00.341046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff468cf-51c7-4417-a74a-2a326d58c869 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.388930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfe20b-6776-4db2-a5f6-25135789cb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.389991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfe20b-6776-4db2-a5f6-25135789cb06 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.390488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfe20b-6776-4db2-a5f6-25135789cb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.390957 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfe20b-6776-4db2-a5f6-25135789cb06 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.400766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcfe20b-6776-4db2-a5f6-25135789cb06 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.402704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcfe20b-6776-4db2-a5f6-25135789cb06 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-bbcfe20b-6776-4db2-a5f6-25135789cb06 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:04:00.417036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:00.418034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:00.419629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:00.419629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:00.419629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:04:00.419629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:04:00.419629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:04:00.419629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:04:00.419629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:04:00.419629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:04:00.419629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:04:00.419629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:00.422296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 459/905] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Aug 30 01:04:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:00.428544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240adf9-e099-4315-b710-0cef604c1fc0 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:00.504503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:00.545017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240adf9-e099-4315-b710-0cef604c1fc0 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.582236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80957e-6040-420d-b843-6477158a9319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.582860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80957e-6040-420d-b843-6477158a9319 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.583137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80957e-6040-420d-b843-6477158a9319 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.583407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80957e-6040-420d-b843-6477158a9319 None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.584175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80957e-6040-420d-b843-6477158a9319 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:00.584625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80957e-6040-420d-b843-6477158a9319 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.586057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80957e-6040-420d-b843-6477158a9319 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.609034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 447/906] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Aug 30 01:04:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:00.617027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfe20b-6776-4db2-a5f6-25135789cb06 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:00.730602 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfe20b-6776-4db2-a5f6-25135789cb06 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.768204 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4762cd-37dc-484a-af95-374f062eaed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.768753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4762cd-37dc-484a-af95-374f062eaed0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.769032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4762cd-37dc-484a-af95-374f062eaed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.769306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4762cd-37dc-484a-af95-374f062eaed0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.783206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:00.846164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4762cd-37dc-484a-af95-374f062eaed0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.847649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4762cd-37dc-484a-af95-374f062eaed0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.956840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 460/907] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:04:00 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:04:00.962276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80957e-6040-420d-b843-6477158a9319 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:01.077917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80957e-6040-420d-b843-6477158a9319 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:01.104871 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:01.117440 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c7224-f9dd-46d8-bbf4-e311bc3060ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:01.118078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c7224-f9dd-46d8-bbf4-e311bc3060ef None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:01.118354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c7224-f9dd-46d8-bbf4-e311bc3060ef None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:01.118623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c7224-f9dd-46d8-bbf4-e311bc3060ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:01.129881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c7224-f9dd-46d8-bbf4-e311bc3060ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:01.131328 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c7224-f9dd-46d8-bbf4-e311bc3060ef None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-430c7224-f9dd-46d8-bbf4-e311bc3060ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:01.147261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:01.148271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:01.149155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:01.149155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:01.149155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:01.149155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 448/908] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:01.154277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4762cd-37dc-484a-af95-374f062eaed0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:01.201832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:01.273532 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4762cd-37dc-484a-af95-374f062eaed0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:01.309671 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d4990-1b0a-42f7-8d2f-86d9e7044e17 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:01.310046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d4990-1b0a-42f7-8d2f-86d9e7044e17 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:01.310279 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d4990-1b0a-42f7-8d2f-86d9e7044e17 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:01.311714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 449/909] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:01.313447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126ce2e-174f-4969-b3c3-01edddd9305d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:01.314072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126ce2e-174f-4969-b3c3-01edddd9305d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:01.314345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126ce2e-174f-4969-b3c3-01edddd9305d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:01.314611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126ce2e-174f-4969-b3c3-01edddd9305d None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:01.315265 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126ce2e-174f-4969-b3c3-01edddd9305d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:01.315565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126ce2e-174f-4969-b3c3-01edddd9305d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:01.317089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126ce2e-174f-4969-b3c3-01edddd9305d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:01.318683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b479d-7971-40b4-8b4d-90e193c152c2 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:01.318971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b479d-7971-40b4-8b4d-90e193c152c2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:01.319246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b479d-7971-40b4-8b4d-90e193c152c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:01.331362 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 461/910] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:01.341801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126ce2e-174f-4969-b3c3-01edddd9305d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:01.347741 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-7e8b479d-7971-40b4-8b4d-90e193c152c2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:01.348986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7e8b479d-7971-40b4-8b4d-90e193c152c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:01.455385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 450/911] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:01.474586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126ce2e-174f-4969-b3c3-01edddd9305d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:01.476473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:01.520588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398d741-eb45-4d85-a7f8-bfbfeb8a45fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:01.521589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398d741-eb45-4d85-a7f8-bfbfeb8a45fc None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:01.521942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398d741-eb45-4d85-a7f8-bfbfeb8a45fc None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:01.522361 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398d741-eb45-4d85-a7f8-bfbfeb8a45fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:01.535161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398d741-eb45-4d85-a7f8-bfbfeb8a45fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:01.537669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398d741-eb45-4d85-a7f8-bfbfeb8a45fc None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-6398d741-eb45-4d85-a7f8-bfbfeb8a45fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:04:01.560370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:01.561446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:01.563252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:04:01.563252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:04:01.563252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:04:01.563252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:04:01.563252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:04:01.563252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:04:01.563252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:01.563252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:01.563252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 462/912] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:04:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:01.572357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b479d-7971-40b4-8b4d-90e193c152c2 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:01.704813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b479d-7971-40b4-8b4d-90e193c152c2 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:01.709490 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:01.751889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a95d4a-fed5-41f4-b321-7a54713c7117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:01.752560 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a95d4a-fed5-41f4-b321-7a54713c7117 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:01.752836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a95d4a-fed5-41f4-b321-7a54713c7117 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:01.753108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a95d4a-fed5-41f4-b321-7a54713c7117 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:01.753814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a95d4a-fed5-41f4-b321-7a54713c7117 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:01.754120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a95d4a-fed5-41f4-b321-7a54713c7117 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:01.755570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a95d4a-fed5-41f4-b321-7a54713c7117 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:01.767809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398d741-eb45-4d85-a7f8-bfbfeb8a45fc None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:01.770321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 451/913] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:04:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:01.776643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a95d4a-fed5-41f4-b321-7a54713c7117 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:01.893322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398d741-eb45-4d85-a7f8-bfbfeb8a45fc None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:01.936487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e1e54-f8b6-4169-9674-a80d48eebf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:01.937131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e1e54-f8b6-4169-9674-a80d48eebf65 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:01.937415 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e1e54-f8b6-4169-9674-a80d48eebf65 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:01.937741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e1e54-f8b6-4169-9674-a80d48eebf65 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:01.938335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e1e54-f8b6-4169-9674-a80d48eebf65 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:01.939804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e1e54-f8b6-4169-9674-a80d48eebf65 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:01.942956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a95d4a-fed5-41f4-b321-7a54713c7117 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-603e1e54-f8b6-4169-9674-a80d48eebf65 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:01.949433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:01.950659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:04:01.951483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:01.955903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 463/914] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:04:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:01.957331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad63729-2723-4009-887a-bb71a9f82b2a None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:01.957618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad63729-2723-4009-887a-bb71a9f82b2a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:01.958608 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad63729-2723-4009-887a-bb71a9f82b2a None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:01.959488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 464/915] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:01.960091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad63729-2723-4009-887a-bb71a9f82b2a None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:01.991806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e926b-c963-4e61-b67d-8aac25ad667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:01.992634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e926b-c963-4e61-b67d-8aac25ad667c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:01.992763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e926b-c963-4e61-b67d-8aac25ad667c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:01.993011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e926b-c963-4e61-b67d-8aac25ad667c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:01.993824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e926b-c963-4e61-b67d-8aac25ad667c None admin] RBAC: Adding query filter params (role.id=71b1513afb19498a9ac3d5584d85750a, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=8a3c3027b61d4f75a67e0a1bb8b52903) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:01.994190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e926b-c963-4e61-b67d-8aac25ad667c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:01.995636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e926b-c963-4e61-b67d-8aac25ad667c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:01.997133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:02.003439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 452/916] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:04:01 2022] GET /identity/v3/role_assignments?user.id=8a3c3027b61d4f75a67e0a1bb8b52903&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=71b1513afb19498a9ac3d5584d85750a => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:02.004941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114fc37-7714-4be4-94c4-a517483d9d94 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:02.005234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114fc37-7714-4be4-94c4-a517483d9d94 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:02.005518 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114fc37-7714-4be4-94c4-a517483d9d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:02.031770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-1114fc37-7714-4be4-94c4-a517483d9d94 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:02.033742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1114fc37-7714-4be4-94c4-a517483d9d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:02.077273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:02.101815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad63729-2723-4009-887a-bb71a9f82b2a None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:02.158875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 453/917] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:02.159847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a055585-2b61-48c4-8b78-8c5fb9a102fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:02.160788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a055585-2b61-48c4-8b78-8c5fb9a102fe None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:02.161233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a055585-2b61-48c4-8b78-8c5fb9a102fe None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:02.161597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a055585-2b61-48c4-8b78-8c5fb9a102fe None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:02.162369 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a055585-2b61-48c4-8b78-8c5fb9a102fe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:02.162862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a055585-2b61-48c4-8b78-8c5fb9a102fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:02.168330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a055585-2b61-48c4-8b78-8c5fb9a102fe None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:02.179303 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 465/918] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:04:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:02.181915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3427c-0502-4cdc-b27e-f52a1d380033 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:02.182189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3427c-0502-4cdc-b27e-f52a1d380033 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:02.182459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3427c-0502-4cdc-b27e-f52a1d380033 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:02.186371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a055585-2b61-48c4-8b78-8c5fb9a102fe None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:02.188440 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 454/919] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:02.190162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7d88f-71ab-4a03-8f42-d93bb165873a None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:02.190534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7d88f-71ab-4a03-8f42-d93bb165873a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:02.190890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7d88f-71ab-4a03-8f42-d93bb165873a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:02.211400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-fce7d88f-71ab-4a03-8f42-d93bb165873a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:02.212692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-fce7d88f-71ab-4a03-8f42-d93bb165873a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:02.319985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 455/920] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:02.321095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:02.333844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a055585-2b61-48c4-8b78-8c5fb9a102fe None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:02.373098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcee3b-753d-452b-99db-56848135f979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:02.373731 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcee3b-753d-452b-99db-56848135f979 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:02.374006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcee3b-753d-452b-99db-56848135f979 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:02.374281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcee3b-753d-452b-99db-56848135f979 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:02.374920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efcee3b-753d-452b-99db-56848135f979 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:02.375223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efcee3b-753d-452b-99db-56848135f979 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:02.376638 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efcee3b-753d-452b-99db-56848135f979 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:02.382510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 466/921] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Aug 30 01:04:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:02.388349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7d88f-71ab-4a03-8f42-d93bb165873a None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:02.443674 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcee3b-753d-452b-99db-56848135f979 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:02.504272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:02.522875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7d88f-71ab-4a03-8f42-d93bb165873a None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:02.571735 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcee3b-753d-452b-99db-56848135f979 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:02.576263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1e83a-e87f-4065-8db9-34c6ff0c16b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:02.576831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1e83a-e87f-4065-8db9-34c6ff0c16b4 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:02.577141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1e83a-e87f-4065-8db9-34c6ff0c16b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:02.577436 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1e83a-e87f-4065-8db9-34c6ff0c16b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:02.578239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1e83a-e87f-4065-8db9-34c6ff0c16b4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:02.579756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1e83a-e87f-4065-8db9-34c6ff0c16b4 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:02.608059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 456/922] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:04:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:04:02.608059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1e83a-e87f-4065-8db9-34c6ff0c16b4 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:02.643053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad07c4-4d63-488e-8044-4cf3286db4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:02.644229 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad07c4-4d63-488e-8044-4cf3286db4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:02.644680 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad07c4-4d63-488e-8044-4cf3286db4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:02.645217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad07c4-4d63-488e-8044-4cf3286db4d2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:02.646030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad07c4-4d63-488e-8044-4cf3286db4d2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:02.647838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad07c4-4d63-488e-8044-4cf3286db4d2 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-a8ad07c4-4d63-488e-8044-4cf3286db4d2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:02.659100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:02.660216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:04:02.661054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:02.664102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 467/923] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Aug 30 01:04:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:02.672504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad07c4-4d63-488e-8044-4cf3286db4d2 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:02.705127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:02.777834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1e83a-e87f-4065-8db9-34c6ff0c16b4 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:02.830542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b25fe5-0fd5-457e-95c0-fcdd7241f02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:02.831632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b25fe5-0fd5-457e-95c0-fcdd7241f02e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:02.833008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b25fe5-0fd5-457e-95c0-fcdd7241f02e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:02.833008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b25fe5-0fd5-457e-95c0-fcdd7241f02e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:02.833636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b25fe5-0fd5-457e-95c0-fcdd7241f02e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:02.835919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b25fe5-0fd5-457e-95c0-fcdd7241f02e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-86b25fe5-0fd5-457e-95c0-fcdd7241f02e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:02.845698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:02.846945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:02.847869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 457/924] 10.222.0.44 () {56 vars in 1132 bytes} [Tue Aug 30 01:04:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:02.853304 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b25fe5-0fd5-457e-95c0-fcdd7241f02e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:02.859028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad07c4-4d63-488e-8044-4cf3286db4d2 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:02.872360 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:02.916086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e057d-6066-48e1-9f4a-8dcbf9723d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:02.917385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e057d-6066-48e1-9f4a-8dcbf9723d56 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:02.917385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e057d-6066-48e1-9f4a-8dcbf9723d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:02.917385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e057d-6066-48e1-9f4a-8dcbf9723d56 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:02.918593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e057d-6066-48e1-9f4a-8dcbf9723d56 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:02.919677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e057d-6066-48e1-9f4a-8dcbf9723d56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:02.920492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e057d-6066-48e1-9f4a-8dcbf9723d56 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:02.929430 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 468/925] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Aug 30 01:04:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:02.935485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e057d-6066-48e1-9f4a-8dcbf9723d56 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:02.997596 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b25fe5-0fd5-457e-95c0-fcdd7241f02e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:03.023217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:03.040917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42b2ef-7d6b-4963-8c61-acf64db26e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:03.041905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42b2ef-7d6b-4963-8c61-acf64db26e4f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:03.042454 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42b2ef-7d6b-4963-8c61-acf64db26e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:03.043781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42b2ef-7d6b-4963-8c61-acf64db26e4f None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:03.044954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42b2ef-7d6b-4963-8c61-acf64db26e4f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:03.045476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42b2ef-7d6b-4963-8c61-acf64db26e4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:03.047196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42b2ef-7d6b-4963-8c61-acf64db26e4f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:03.050567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e057d-6066-48e1-9f4a-8dcbf9723d56 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:03.053898 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 458/926] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Aug 30 01:04:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:03.059976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42b2ef-7d6b-4963-8c61-acf64db26e4f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:03.102601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ddc2b-1654-4e98-bc09-135303fe43bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:03.103580 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ddc2b-1654-4e98-bc09-135303fe43bc None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:03.104076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ddc2b-1654-4e98-bc09-135303fe43bc None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:03.104532 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ddc2b-1654-4e98-bc09-135303fe43bc None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:03.105425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ddc2b-1654-4e98-bc09-135303fe43bc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:03.106470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ddc2b-1654-4e98-bc09-135303fe43bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:03.108101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ddc2b-1654-4e98-bc09-135303fe43bc None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:03.115573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 469/927] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Aug 30 01:04:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:03.124431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ddc2b-1654-4e98-bc09-135303fe43bc None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:03.201897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:03.250846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42b2ef-7d6b-4963-8c61-acf64db26e4f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:03.281864 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ddc2b-1654-4e98-bc09-135303fe43bc None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:03.289805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f079602-d787-4759-bede-b135f7070a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:03.290566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f079602-d787-4759-bede-b135f7070a49 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:03.291001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f079602-d787-4759-bede-b135f7070a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:03.291321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f079602-d787-4759-bede-b135f7070a49 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:03.292344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f079602-d787-4759-bede-b135f7070a49 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:03.292602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f079602-d787-4759-bede-b135f7070a49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:03.295135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f079602-d787-4759-bede-b135f7070a49 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:03.305732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 459/928] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Aug 30 01:04:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:03.306755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f079602-d787-4759-bede-b135f7070a49 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:03.330297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3424fb1-91a0-4362-afff-2d774c472cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:03.331196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3424fb1-91a0-4362-afff-2d774c472cba None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:03.331619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3424fb1-91a0-4362-afff-2d774c472cba None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:03.332074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3424fb1-91a0-4362-afff-2d774c472cba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:03.333486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3424fb1-91a0-4362-afff-2d774c472cba None admin] RBAC: Adding query filter params (interface=public, service_id=a8272e2f35444ffa9a6fbe8a9acf4735, region_id=RegionOne) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:03.333953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3424fb1-91a0-4362-afff-2d774c472cba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:03.335769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3424fb1-91a0-4362-afff-2d774c472cba None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:03.344943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 470/929] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Aug 30 01:04:03 2022] GET /identity/v3/endpoints?service_id=a8272e2f35444ffa9a6fbe8a9acf4735&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:03.350461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3424fb1-91a0-4362-afff-2d774c472cba None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:03.404367 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:03.445619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f079602-d787-4759-bede-b135f7070a49 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:03.483662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3424fb1-91a0-4362-afff-2d774c472cba None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:03.500115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cd56b-a7e8-4cd7-b1e9-bf51821d6ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:03.501628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cd56b-a7e8-4cd7-b1e9-bf51821d6ca9 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:03.502129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cd56b-a7e8-4cd7-b1e9-bf51821d6ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:03.502784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cd56b-a7e8-4cd7-b1e9-bf51821d6ca9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:03.503644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212cd56b-a7e8-4cd7-b1e9-bf51821d6ca9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:03.506237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212cd56b-a7e8-4cd7-b1e9-bf51821d6ca9 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:03.521671 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:03.536703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a435777-3025-4643-a59e-9b873e9039fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:03.537929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a435777-3025-4643-a59e-9b873e9039fa None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:03.538478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a435777-3025-4643-a59e-9b873e9039fa None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:03.539023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a435777-3025-4643-a59e-9b873e9039fa None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:03.539988 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a435777-3025-4643-a59e-9b873e9039fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:03.548154 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 460/930] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:04:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:04:03.549080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a435777-3025-4643-a59e-9b873e9039fa None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:03.558055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 471/931] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Aug 30 01:04:03 2022] GET /identity/v3/services => generated 1497 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:04:04.105330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adf7f1-8c1b-49c3-937c-688b5f26cb39 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:04.105909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adf7f1-8c1b-49c3-937c-688b5f26cb39 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:04.106339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adf7f1-8c1b-49c3-937c-688b5f26cb39 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:04.108566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 461/932] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:04.115352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710d12c-31fc-49ff-aad9-b0d68c1a3bdb None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:04.116567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710d12c-31fc-49ff-aad9-b0d68c1a3bdb None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:04.116567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710d12c-31fc-49ff-aad9-b0d68c1a3bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:04.136246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e710d12c-31fc-49ff-aad9-b0d68c1a3bdb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:04.138764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e710d12c-31fc-49ff-aad9-b0d68c1a3bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:04.216523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:04.229324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 472/933] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:04.500290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75adf7f1-8c1b-49c3-937c-688b5f26cb39 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:04.648635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75adf7f1-8c1b-49c3-937c-688b5f26cb39 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:04.688492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58fa90b-75ec-4ab9-8b89-c63ed7a643f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:04.689143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fa90b-75ec-4ab9-8b89-c63ed7a643f3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:04.689419 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fa90b-75ec-4ab9-8b89-c63ed7a643f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:04.689688 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fa90b-75ec-4ab9-8b89-c63ed7a643f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:04.703544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58fa90b-75ec-4ab9-8b89-c63ed7a643f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-f58fa90b-75ec-4ab9-8b89-c63ed7a643f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:04:04.708459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:04.709397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:04.710382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 462/934] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Aug 30 01:04:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:04.718540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710d12c-31fc-49ff-aad9-b0d68c1a3bdb None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:04.805287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a1807-e18c-4789-95df-b9613c7b3c36 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:04.805681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a1807-e18c-4789-95df-b9613c7b3c36 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:04.806140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a1807-e18c-4789-95df-b9613c7b3c36 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:04.808655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 463/935] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:04.814656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192318f-5e9b-40a5-9481-8aaff1a50b41 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:04.815001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192318f-5e9b-40a5-9481-8aaff1a50b41 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:04.815246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192318f-5e9b-40a5-9481-8aaff1a50b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:04.834916 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-6192318f-5e9b-40a5-9481-8aaff1a50b41 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:04.836268 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6192318f-5e9b-40a5-9481-8aaff1a50b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:04.842836 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710d12c-31fc-49ff-aad9-b0d68c1a3bdb None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:04.888855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214769c3-07fb-4338-a1f1-0f1188a9913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:04.890306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214769c3-07fb-4338-a1f1-0f1188a9913b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:04.891073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214769c3-07fb-4338-a1f1-0f1188a9913b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:04.891812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214769c3-07fb-4338-a1f1-0f1188a9913b None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:04.893180 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214769c3-07fb-4338-a1f1-0f1188a9913b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:04.893632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:04.894264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214769c3-07fb-4338-a1f1-0f1188a9913b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:04.897492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214769c3-07fb-4338-a1f1-0f1188a9913b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:04.913603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 473/936] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Aug 30 01:04:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:04.915778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214769c3-07fb-4338-a1f1-0f1188a9913b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:04.932155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 464/937] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:04.979014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:05.029071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214769c3-07fb-4338-a1f1-0f1188a9913b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:05.066163 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30cd07-03e0-47b2-9fab-01bb8d67f9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:05.066762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30cd07-03e0-47b2-9fab-01bb8d67f9fc None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:05.067051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30cd07-03e0-47b2-9fab-01bb8d67f9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:05.067320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30cd07-03e0-47b2-9fab-01bb8d67f9fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:05.078979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30cd07-03e0-47b2-9fab-01bb8d67f9fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:05.080563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30cd07-03e0-47b2-9fab-01bb8d67f9fc None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-dc30cd07-03e0-47b2-9fab-01bb8d67f9fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:05.096084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:05.097289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:05.098232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:05.098232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:05.098232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:05.102299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 474/938] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:05.105250 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192318f-5e9b-40a5-9481-8aaff1a50b41 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:05.215218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192318f-5e9b-40a5-9481-8aaff1a50b41 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:05.222648 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30cd07-03e0-47b2-9fab-01bb8d67f9fc None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:05.262836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670474ef-3479-496d-b1aa-b0630f39c5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:05.263471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670474ef-3479-496d-b1aa-b0630f39c5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:05.264805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670474ef-3479-496d-b1aa-b0630f39c5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:05.265074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670474ef-3479-496d-b1aa-b0630f39c5a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:05.265735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670474ef-3479-496d-b1aa-b0630f39c5a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:05.266038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670474ef-3479-496d-b1aa-b0630f39c5a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:05.267472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670474ef-3479-496d-b1aa-b0630f39c5a3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:05.288070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 465/939] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:05.290550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670474ef-3479-496d-b1aa-b0630f39c5a3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:05.350799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30cd07-03e0-47b2-9fab-01bb8d67f9fc None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:05.354706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:05.363795 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:05.398278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea39fb-fd85-466f-80b6-cb545a78af94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:05.398939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea39fb-fd85-466f-80b6-cb545a78af94 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:05.399229 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea39fb-fd85-466f-80b6-cb545a78af94 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:05.399499 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea39fb-fd85-466f-80b6-cb545a78af94 None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:05.400265 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea39fb-fd85-466f-80b6-cb545a78af94 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:05.401726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea39fb-fd85-466f-80b6-cb545a78af94 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:05.405294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: WARNING keystone.server.flask.common [None req-60ea39fb-fd85-466f-80b6-cb545a78af94 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 01:04:05.425523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.sql.core [None req-60ea39fb-fd85-466f-80b6-cb545a78af94 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 01:04:05.425523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 01:04:05.425523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [parameters: {'id': '94c4c1111d1e46b2b7c78cc7858e3c39', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 01:04:05.425523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81647) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-60ea39fb-fd85-466f-80b6-cb545a78af94 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94c4c1111d1e46b2b7c78cc7858e3c39.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94c4c1111d1e46b2b7c78cc7858e3c39. Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application conn.query(q) Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 01:04:05.433356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result.read() Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 30 01:04:05.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application next(self.gen) Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application yield resource Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application next(self.gen) Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise self.value Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application session.commit() Aug 30 01:04:05.435247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.session.flush() Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application compat.raise_( Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 01:04:05.436819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 01:04:05.437792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application conn.query(q) Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result.read() Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 01:04:05.438574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [parameters: {'id': '94c4c1111d1e46b2b7c78cc7858e3c39', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:05.439440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:05.442162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:05.444452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:05.444452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 30 01:04:05.444452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 30 01:04:05.444452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:05.444452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:05.444452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 30 01:04:05.444452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 30 01:04:05.444452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94c4c1111d1e46b2b7c78cc7858e3c39. Aug 30 01:04:05.444452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:05.444452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 475/940] 10.222.0.44 () {60 vars in 1140 bytes} [Tue Aug 30 01:04:05 2022] POST /identity/v3/projects => generated 262 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:05.445782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670474ef-3479-496d-b1aa-b0630f39c5a3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:05.445963 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea39fb-fd85-466f-80b6-cb545a78af94 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:05.496438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc019fb-f313-44f8-a318-f9f2b0a5da49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:05.498533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc019fb-f313-44f8-a318-f9f2b0a5da49 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:05.499116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc019fb-f313-44f8-a318-f9f2b0a5da49 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:05.499592 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc019fb-f313-44f8-a318-f9f2b0a5da49 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:05.511175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc019fb-f313-44f8-a318-f9f2b0a5da49 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:05.514262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc019fb-f313-44f8-a318-f9f2b0a5da49 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-3bc019fb-f313-44f8-a318-f9f2b0a5da49 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:04:05.533367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:05.535284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:05.537732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:05.537732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:05.537732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:04:05.537732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:04:05.537732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:04:05.537732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:04:05.537732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:04:05.537732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:04:05.537732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:04:05.537732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:05.540193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 466/941] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Aug 30 01:04:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:05.541073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc019fb-f313-44f8-a318-f9f2b0a5da49 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:05.617141 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea39fb-fd85-466f-80b6-cb545a78af94 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:05.654122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872df150-a078-4e50-aa96-16231f8dd227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:05.654815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872df150-a078-4e50-aa96-16231f8dd227 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:05.655025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872df150-a078-4e50-aa96-16231f8dd227 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:05.655288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872df150-a078-4e50-aa96-16231f8dd227 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:05.662431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872df150-a078-4e50-aa96-16231f8dd227 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:05.664124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872df150-a078-4e50-aa96-16231f8dd227 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:05.664260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc019fb-f313-44f8-a318-f9f2b0a5da49 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:05.666085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-872df150-a078-4e50-aa96-16231f8dd227 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:04:05.677805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:05.678814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:05.679787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:04:05.679787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:04:05.679787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:04:05.679787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:04:05.679787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:04:05.679787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:04:05.679787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:05.679787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:05.679787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 476/942] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:04:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:05.681030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4dc1d-806b-43bd-ac43-7f83a61cb24f None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:05.681030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4dc1d-806b-43bd-ac43-7f83a61cb24f None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:05.681030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4dc1d-806b-43bd-ac43-7f83a61cb24f None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:05.685040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 477/943] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:05.689455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4dc1d-806b-43bd-ac43-7f83a61cb24f None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:05.719878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff10346-2f3b-4f4c-a80c-3e45822d7ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:05.720528 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff10346-2f3b-4f4c-a80c-3e45822d7ddd None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:05.720804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff10346-2f3b-4f4c-a80c-3e45822d7ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:05.721071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff10346-2f3b-4f4c-a80c-3e45822d7ddd None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:05.721820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff10346-2f3b-4f4c-a80c-3e45822d7ddd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:05.722124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff10346-2f3b-4f4c-a80c-3e45822d7ddd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:05.723656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff10346-2f3b-4f4c-a80c-3e45822d7ddd None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:05.727088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:05.752151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 467/944] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Aug 30 01:04:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:05.752151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f2e29-e682-4cfb-98d9-11076116aa10 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:05.752151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f2e29-e682-4cfb-98d9-11076116aa10 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:05.752151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f2e29-e682-4cfb-98d9-11076116aa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:05.768269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c54f2e29-e682-4cfb-98d9-11076116aa10 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:05.769505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c54f2e29-e682-4cfb-98d9-11076116aa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:05.842553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4dc1d-806b-43bd-ac43-7f83a61cb24f None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:05.873715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 468/945] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:05.873715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f2e29-e682-4cfb-98d9-11076116aa10 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:05.898008 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eae991-e8c6-4998-80d1-b4b50090211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:05.898734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eae991-e8c6-4998-80d1-b4b50090211a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:05.899042 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eae991-e8c6-4998-80d1-b4b50090211a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:05.899289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eae991-e8c6-4998-80d1-b4b50090211a None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:05.899959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eae991-e8c6-4998-80d1-b4b50090211a None admin] RBAC: Adding query filter params (name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:05.900302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eae991-e8c6-4998-80d1-b4b50090211a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:05.901766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eae991-e8c6-4998-80d1-b4b50090211a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:05.921189 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 478/946] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:04:05 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:05.922676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13837bb5-88a2-4111-a15b-2be37f9fb321 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:05.922967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13837bb5-88a2-4111-a15b-2be37f9fb321 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:05.923307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13837bb5-88a2-4111-a15b-2be37f9fb321 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:05.929365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 479/947] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:05.930930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842274e-a3d7-4025-8ad6-40d8cb731346 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:05.931235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842274e-a3d7-4025-8ad6-40d8cb731346 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:05.931486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842274e-a3d7-4025-8ad6-40d8cb731346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:05.953922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d842274e-a3d7-4025-8ad6-40d8cb731346 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:05.955601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d842274e-a3d7-4025-8ad6-40d8cb731346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:06.026334 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:06.048801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f2e29-e682-4cfb-98d9-11076116aa10 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:06.066677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 480/948] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:06.104415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb3b78-0bfb-4aea-92e5-336e4994f131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:06.106013 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb3b78-0bfb-4aea-92e5-336e4994f131 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:06.106604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb3b78-0bfb-4aea-92e5-336e4994f131 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:06.107089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb3b78-0bfb-4aea-92e5-336e4994f131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:06.121523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb3b78-0bfb-4aea-92e5-336e4994f131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:06.122992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb3b78-0bfb-4aea-92e5-336e4994f131 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-f5fb3b78-0bfb-4aea-92e5-336e4994f131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:06.148810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:06.149884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:06.150879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:06.150879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:06.150879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:06.151468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 469/949] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:06.155944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842274e-a3d7-4025-8ad6-40d8cb731346 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:06.158989 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb3b78-0bfb-4aea-92e5-336e4994f131 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:06.196199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:06.293790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb3b78-0bfb-4aea-92e5-336e4994f131 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:06.332213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94223818-aa1c-4a7b-8fdf-8453cabea17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:06.332831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94223818-aa1c-4a7b-8fdf-8453cabea17f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:06.333123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94223818-aa1c-4a7b-8fdf-8453cabea17f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:06.333377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94223818-aa1c-4a7b-8fdf-8453cabea17f None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:06.334021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94223818-aa1c-4a7b-8fdf-8453cabea17f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:06.334329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94223818-aa1c-4a7b-8fdf-8453cabea17f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:06.335815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94223818-aa1c-4a7b-8fdf-8453cabea17f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:06.351525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 470/950] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:06.352519 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94223818-aa1c-4a7b-8fdf-8453cabea17f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:06.355474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:06.368076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842274e-a3d7-4025-8ad6-40d8cb731346 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:06.418578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2ecb1-cc30-495a-8a85-789d028b6de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:06.420012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2ecb1-cc30-495a-8a85-789d028b6de0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:06.420433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2ecb1-cc30-495a-8a85-789d028b6de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:06.420986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2ecb1-cc30-495a-8a85-789d028b6de0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:06.421596 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2ecb1-cc30-495a-8a85-789d028b6de0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:06.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2ecb1-cc30-495a-8a85-789d028b6de0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-15d2ecb1-cc30-495a-8a85-789d028b6de0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:06.436713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:06.438027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:04:06.439573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:06.444134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 481/951] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:04:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:06.445120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2ecb1-cc30-495a-8a85-789d028b6de0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:06.476785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94223818-aa1c-4a7b-8fdf-8453cabea17f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:06.522317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e63d66-b811-4b26-a013-5734b211597d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:06.523015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e63d66-b811-4b26-a013-5734b211597d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:06.523371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e63d66-b811-4b26-a013-5734b211597d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:06.523737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e63d66-b811-4b26-a013-5734b211597d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:06.524620 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e63d66-b811-4b26-a013-5734b211597d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:06.526182 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e63d66-b811-4b26-a013-5734b211597d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-10e63d66-b811-4b26-a013-5734b211597d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:06.541758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:06.543075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:04:06.544260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:06.548544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 471/952] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Aug 30 01:04:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:06.551711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e63d66-b811-4b26-a013-5734b211597d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:06.569001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:06.597687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2ecb1-cc30-495a-8a85-789d028b6de0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:06.644510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:06.653131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec87a36-5057-4388-8e49-f48cd760b06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:06.653966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec87a36-5057-4388-8e49-f48cd760b06e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:06.654091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec87a36-5057-4388-8e49-f48cd760b06e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:06.654369 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec87a36-5057-4388-8e49-f48cd760b06e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:06.661806 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec87a36-5057-4388-8e49-f48cd760b06e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:06.663280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec87a36-5057-4388-8e49-f48cd760b06e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-2ec87a36-5057-4388-8e49-f48cd760b06e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:04:06.676281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:06.677272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:06.678248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:04:06.678248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:04:06.678248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:04:06.678248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:04:06.678248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:04:06.678248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:04:06.678248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:06.678248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:06.678248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 482/953] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:04:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:06.678740 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec87a36-5057-4388-8e49-f48cd760b06e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:06.695437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e63d66-b811-4b26-a013-5734b211597d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:06.740835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa44f3-ac20-4367-8cc9-1cf8c1d53672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:06.741488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa44f3-ac20-4367-8cc9-1cf8c1d53672 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:06.741806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa44f3-ac20-4367-8cc9-1cf8c1d53672 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:06.742082 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa44f3-ac20-4367-8cc9-1cf8c1d53672 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:06.742715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa44f3-ac20-4367-8cc9-1cf8c1d53672 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:06.743023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa44f3-ac20-4367-8cc9-1cf8c1d53672 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:06.744429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa44f3-ac20-4367-8cc9-1cf8c1d53672 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:06.749865 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 472/954] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:04:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:06.750729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa44f3-ac20-4367-8cc9-1cf8c1d53672 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:06.812692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec87a36-5057-4388-8e49-f48cd760b06e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:06.851374 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dfb6e-0397-457b-a3b7-1a258084b115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:06.852017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dfb6e-0397-457b-a3b7-1a258084b115 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:06.852297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dfb6e-0397-457b-a3b7-1a258084b115 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:06.852578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dfb6e-0397-457b-a3b7-1a258084b115 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:06.853271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00dfb6e-0397-457b-a3b7-1a258084b115 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:06.853557 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00dfb6e-0397-457b-a3b7-1a258084b115 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:06.855290 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00dfb6e-0397-457b-a3b7-1a258084b115 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:06.864202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 483/955] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Aug 30 01:04:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:06.865767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dfb6e-0397-457b-a3b7-1a258084b115 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:06.867348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa44f3-ac20-4367-8cc9-1cf8c1d53672 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:06.873082 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:06.912263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:06.922477 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b375a65-9a9b-4ef9-9f0e-ed9b9a783498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:06.923387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b375a65-9a9b-4ef9-9f0e-ed9b9a783498 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:06.923911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b375a65-9a9b-4ef9-9f0e-ed9b9a783498 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:06.924595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b375a65-9a9b-4ef9-9f0e-ed9b9a783498 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:06.926054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b375a65-9a9b-4ef9-9f0e-ed9b9a783498 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:06.926813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b375a65-9a9b-4ef9-9f0e-ed9b9a783498 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:06.929354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b375a65-9a9b-4ef9-9f0e-ed9b9a783498 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:06.952049 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 473/956] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:04:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:06.952049 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b375a65-9a9b-4ef9-9f0e-ed9b9a783498 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:06.995182 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dfb6e-0397-457b-a3b7-1a258084b115 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:07.052537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca180b0c-8a29-4675-b495-537ebb29911f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:07.053264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca180b0c-8a29-4675-b495-537ebb29911f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:07.053549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca180b0c-8a29-4675-b495-537ebb29911f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:07.053989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca180b0c-8a29-4675-b495-537ebb29911f None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:07.055206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca180b0c-8a29-4675-b495-537ebb29911f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:07.055724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca180b0c-8a29-4675-b495-537ebb29911f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:07.058212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca180b0c-8a29-4675-b495-537ebb29911f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:07.076064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 484/957] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:04:06 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:07.076064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca180b0c-8a29-4675-b495-537ebb29911f None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:07.114602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b375a65-9a9b-4ef9-9f0e-ed9b9a783498 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:07.187503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:07.189023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1d47b-e179-4b75-a7e5-dd371f0c820e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:07.189933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1d47b-e179-4b75-a7e5-dd371f0c820e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:07.190942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1d47b-e179-4b75-a7e5-dd371f0c820e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:07.191469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1d47b-e179-4b75-a7e5-dd371f0c820e None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:07.192316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1d47b-e179-4b75-a7e5-dd371f0c820e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:07.193427 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1d47b-e179-4b75-a7e5-dd371f0c820e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:07.196223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1d47b-e179-4b75-a7e5-dd371f0c820e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:07.202904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 474/958] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Aug 30 01:04:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:07.209359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1d47b-e179-4b75-a7e5-dd371f0c820e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:07.247187 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca180b0c-8a29-4675-b495-537ebb29911f None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:07.294282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f265c8-52e1-47c6-9ed6-f4b442d08b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:07.295279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f265c8-52e1-47c6-9ed6-f4b442d08b22 None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:07.295562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f265c8-52e1-47c6-9ed6-f4b442d08b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:07.295837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f265c8-52e1-47c6-9ed6-f4b442d08b22 None admin] PATH_INFO: `/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/8a3c3027b61d4f75a67e0a1bb8b52903/roles/71b1513afb19498a9ac3d5584d85750a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:07.296796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:07.300688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f265c8-52e1-47c6-9ed6-f4b442d08b22 None admin] RBAC: Authorizing `identity:create_grant(project_id=9188023bf51d4737b801e3fbd9af95b1, user_id=8a3c3027b61d4f75a67e0a1bb8b52903, role_id=71b1513afb19498a9ac3d5584d85750a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:07.302578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f265c8-52e1-47c6-9ed6-f4b442d08b22 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:07.319428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 485/959] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:04:07 2022] PUT /identity/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/8a3c3027b61d4f75a67e0a1bb8b52903/roles/71b1513afb19498a9ac3d5584d85750a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:04:07.350710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1d47b-e179-4b75-a7e5-dd371f0c820e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:07.392462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da3cb7-3c1e-454c-92a4-0a6bda0bec5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:07.393174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da3cb7-3c1e-454c-92a4-0a6bda0bec5e None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:07.393455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da3cb7-3c1e-454c-92a4-0a6bda0bec5e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:07.393723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da3cb7-3c1e-454c-92a4-0a6bda0bec5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:07.394397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da3cb7-3c1e-454c-92a4-0a6bda0bec5e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:07.395888 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da3cb7-3c1e-454c-92a4-0a6bda0bec5e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:07.419960 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 475/960] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:04:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:04:08.170143 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297a9bb-ac91-4e7b-a9eb-99ef7eb3eb6c None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:08.170663 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297a9bb-ac91-4e7b-a9eb-99ef7eb3eb6c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:08.170663 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297a9bb-ac91-4e7b-a9eb-99ef7eb3eb6c None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:08.172435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 486/961] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:08.178091 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdeb1d7-3e66-4ee5-bd94-2bf3328c80d2 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:08.178384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdeb1d7-3e66-4ee5-bd94-2bf3328c80d2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:08.178638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdeb1d7-3e66-4ee5-bd94-2bf3328c80d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:08.198624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-2fdeb1d7-3e66-4ee5-bd94-2bf3328c80d2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:08.200981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2fdeb1d7-3e66-4ee5-bd94-2bf3328c80d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:08.294064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 476/962] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:08.577752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297a9bb-ac91-4e7b-a9eb-99ef7eb3eb6c None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:08.690814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297a9bb-ac91-4e7b-a9eb-99ef7eb3eb6c None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:08.720561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:08.730696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7b79e-edb5-44b3-acbe-b96b813993ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:08.731577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7b79e-edb5-44b3-acbe-b96b813993ef None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:08.731857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7b79e-edb5-44b3-acbe-b96b813993ef None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:08.732161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7b79e-edb5-44b3-acbe-b96b813993ef None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:08.732825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7b79e-edb5-44b3-acbe-b96b813993ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:08.734260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7b79e-edb5-44b3-acbe-b96b813993ef None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:08.751245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.sql.core [None req-50a7b79e-edb5-44b3-acbe-b96b813993ef None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 01:04:08.751245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 01:04:08.751245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [parameters: {'user_id': '97159eab5afc48aaa77d1f7eb5367f44', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 01:04:08.751245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81647) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-50a7b79e-edb5-44b3-acbe-b96b813993ef None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application conn.query(q) Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result.read() Aug 30 01:04:08.758940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 01:04:08.761166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application next(self.gen) Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application yield resource Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application next(self.gen) Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise self.value Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application session.commit() Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 01:04:08.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.session.flush() Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application compat.raise_( Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 01:04:08.765894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 01:04:08.768232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application conn.query(q) Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application result.read() Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 01:04:08.770291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [parameters: {'user_id': '97159eab5afc48aaa77d1f7eb5367f44', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:08.772463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 30 01:04:08.774522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 30 01:04:08.776948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 30 01:04:08.776948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 30 01:04:08.776948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 01:04:08.776948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 01:04:08.776948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 01:04:08.776948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:08.776948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 487/963] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Aug 30 01:04:08 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:08.778406 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdeb1d7-3e66-4ee5-bd94-2bf3328c80d2 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:08.816078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:08.886277 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdeb1d7-3e66-4ee5-bd94-2bf3328c80d2 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:08.927022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a2af7-1a63-412d-915b-4f7dbbe76e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:08.928138 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a2af7-1a63-412d-915b-4f7dbbe76e9f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:08.928637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a2af7-1a63-412d-915b-4f7dbbe76e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:08.929111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a2af7-1a63-412d-915b-4f7dbbe76e9f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:08.937343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a2af7-1a63-412d-915b-4f7dbbe76e9f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:08.939057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a2af7-1a63-412d-915b-4f7dbbe76e9f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-d30a2af7-1a63-412d-915b-4f7dbbe76e9f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:04:08.953634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:08.954626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:08.955587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:08.955587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:08.955587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:04:08.955587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:04:08.955587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:04:08.955587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:04:08.955587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:04:08.955587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:04:08.955587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:04:08.955587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:08.958181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 477/964] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:04:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:08.964059 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7b79e-edb5-44b3-acbe-b96b813993ef None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:09.080371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7b79e-edb5-44b3-acbe-b96b813993ef None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:09.117345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f635855-aa5f-4105-90d4-5a325982c4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:09.117962 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f635855-aa5f-4105-90d4-5a325982c4fd None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:09.118241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f635855-aa5f-4105-90d4-5a325982c4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:09.118510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f635855-aa5f-4105-90d4-5a325982c4fd None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:09.119208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f635855-aa5f-4105-90d4-5a325982c4fd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:09.119508 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f635855-aa5f-4105-90d4-5a325982c4fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:09.121936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f635855-aa5f-4105-90d4-5a325982c4fd None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:09.168779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 488/965] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Aug 30 01:04:08 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:09.178805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0fd77-c5bf-47d3-8073-db9d3a4faaf2 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:09.179146 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0fd77-c5bf-47d3-8073-db9d3a4faaf2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:09.179350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0fd77-c5bf-47d3-8073-db9d3a4faaf2 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:09.181133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 478/966] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:09.187655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49790581-fd43-49fc-ab21-fb065bbd977b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:09.188201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49790581-fd43-49fc-ab21-fb065bbd977b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:09.188278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49790581-fd43-49fc-ab21-fb065bbd977b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:09.210140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-49790581-fd43-49fc-ab21-fb065bbd977b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:09.212317 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-49790581-fd43-49fc-ab21-fb065bbd977b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:09.263723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:09.294727 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 489/967] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:09.449956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf772b8-afb6-4e28-967c-b65618366d75 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:09.450338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf772b8-afb6-4e28-967c-b65618366d75 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:09.450573 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf772b8-afb6-4e28-967c-b65618366d75 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:09.452255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 479/968] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:09.457856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd51fa-d85d-461e-99ee-b650a5b65bf5 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:09.458144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd51fa-d85d-461e-99ee-b650a5b65bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:09.458481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd51fa-d85d-461e-99ee-b650a5b65bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:09.482978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-87bd51fa-d85d-461e-99ee-b650a5b65bf5 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:09.484214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-87bd51fa-d85d-461e-99ee-b650a5b65bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:09.571142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 490/969] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:09.578850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf772b8-afb6-4e28-967c-b65618366d75 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:09.738116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:09.746915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf772b8-afb6-4e28-967c-b65618366d75 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:09.789402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456da5f-49d1-411b-ab11-cf4187f1ce61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:09.790189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456da5f-49d1-411b-ab11-cf4187f1ce61 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:09.790315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456da5f-49d1-411b-ab11-cf4187f1ce61 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:09.790570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456da5f-49d1-411b-ab11-cf4187f1ce61 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:09.791185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456da5f-49d1-411b-ab11-cf4187f1ce61 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:09.792613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456da5f-49d1-411b-ab11-cf4187f1ce61 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:09.816048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 480/970] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:04:09 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:04:09.830454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb4182-291a-41a0-ab1d-eb43c7780119 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:09.830791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb4182-291a-41a0-ab1d-eb43c7780119 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:09.831004 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb4182-291a-41a0-ab1d-eb43c7780119 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:09.832596 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 491/971] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:09.839564 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d6416-a408-46af-a1e5-44a022bc4ae2 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:09.839975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d6416-a408-46af-a1e5-44a022bc4ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:09.840258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d6416-a408-46af-a1e5-44a022bc4ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:09.848284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb4182-291a-41a0-ab1d-eb43c7780119 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:09.863968 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-834d6416-a408-46af-a1e5-44a022bc4ae2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:09.865257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-834d6416-a408-46af-a1e5-44a022bc4ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:09.969459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 481/972] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:10.001966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb4182-291a-41a0-ab1d-eb43c7780119 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:10.014241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:10.048562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9907ed6-0e73-48d8-90b5-7f2f72d83218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:10.049205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9907ed6-0e73-48d8-90b5-7f2f72d83218 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:10.049480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9907ed6-0e73-48d8-90b5-7f2f72d83218 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:10.049751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9907ed6-0e73-48d8-90b5-7f2f72d83218 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:10.062958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9907ed6-0e73-48d8-90b5-7f2f72d83218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-e9907ed6-0e73-48d8-90b5-7f2f72d83218 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:04:10.068100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:10.069107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:10.072831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 492/973] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Aug 30 01:04:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:10.076446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d6416-a408-46af-a1e5-44a022bc4ae2 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:10.188853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:10.212423 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d6416-a408-46af-a1e5-44a022bc4ae2 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:10.242171 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9907ed6-0e73-48d8-90b5-7f2f72d83218 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:10.258041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d414a-d8b3-4fd6-9245-dc4369bc48d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:10.259077 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d414a-d8b3-4fd6-9245-dc4369bc48d1 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:10.259354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d414a-d8b3-4fd6-9245-dc4369bc48d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:10.259795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d414a-d8b3-4fd6-9245-dc4369bc48d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:10.260542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d414a-d8b3-4fd6-9245-dc4369bc48d1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:10.260848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d414a-d8b3-4fd6-9245-dc4369bc48d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:10.262621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d414a-d8b3-4fd6-9245-dc4369bc48d1 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:10.272475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 482/974] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Aug 30 01:04:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:10.278863 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d414a-d8b3-4fd6-9245-dc4369bc48d1 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:10.399386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d414a-d8b3-4fd6-9245-dc4369bc48d1 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:10.409116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9907ed6-0e73-48d8-90b5-7f2f72d83218 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:10.450857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990930fa-de3d-4a55-9bbf-17db1738a2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:10.451965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990930fa-de3d-4a55-9bbf-17db1738a2cb None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:10.452284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990930fa-de3d-4a55-9bbf-17db1738a2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:10.452554 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990930fa-de3d-4a55-9bbf-17db1738a2cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:10.458158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e313ca-dfbb-4d19-9c4f-284a67661ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:10.459159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e313ca-dfbb-4d19-9c4f-284a67661ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:10.459637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e313ca-dfbb-4d19-9c4f-284a67661ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:10.460160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e313ca-dfbb-4d19-9c4f-284a67661ec1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:10.460950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e313ca-dfbb-4d19-9c4f-284a67661ec1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:10.463685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e313ca-dfbb-4d19-9c4f-284a67661ec1 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:10.465747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990930fa-de3d-4a55-9bbf-17db1738a2cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:10.467913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990930fa-de3d-4a55-9bbf-17db1738a2cb None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:10.472865 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-53e313ca-dfbb-4d19-9c4f-284a67661ec1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:10.476777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:10.477773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:04:10.478618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:10.487352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 493/975] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Aug 30 01:04:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:10.488028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e313ca-dfbb-4d19-9c4f-284a67661ec1 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-990930fa-de3d-4a55-9bbf-17db1738a2cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:10.496886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:10.498003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:10.498995 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:10.498995 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:10.498995 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:10.504895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 483/976] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:10.505924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990930fa-de3d-4a55-9bbf-17db1738a2cb None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:10.542193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:10.625635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e313ca-dfbb-4d19-9c4f-284a67661ec1 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:10.665319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990930fa-de3d-4a55-9bbf-17db1738a2cb None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:10.693938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fcd68-78b1-42f0-90ff-6ff8f24c87c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:10.694582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fcd68-78b1-42f0-90ff-6ff8f24c87c6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:10.694926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fcd68-78b1-42f0-90ff-6ff8f24c87c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:10.695207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fcd68-78b1-42f0-90ff-6ff8f24c87c6 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:10.695760 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fcd68-78b1-42f0-90ff-6ff8f24c87c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:10.699876 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fcd68-78b1-42f0-90ff-6ff8f24c87c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:10.701358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fcd68-78b1-42f0-90ff-6ff8f24c87c6 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:10.712230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 494/977] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Aug 30 01:04:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:10.716985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fcd68-78b1-42f0-90ff-6ff8f24c87c6 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:10.740854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c2a12-d16b-4e85-a8ab-e240c3b8cc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:10.741492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c2a12-d16b-4e85-a8ab-e240c3b8cc70 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:10.741770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c2a12-d16b-4e85-a8ab-e240c3b8cc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:10.742252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c2a12-d16b-4e85-a8ab-e240c3b8cc70 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:10.743257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c2a12-d16b-4e85-a8ab-e240c3b8cc70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:10.743566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c2a12-d16b-4e85-a8ab-e240c3b8cc70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:10.744978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c2a12-d16b-4e85-a8ab-e240c3b8cc70 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:10.760149 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 484/978] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:10.766055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c2a12-d16b-4e85-a8ab-e240c3b8cc70 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:10.824481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:10.831776 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:10.834159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fcd68-78b1-42f0-90ff-6ff8f24c87c6 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:10.876418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6992eda-df05-488e-83f4-faa77beeab16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:10.877024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6992eda-df05-488e-83f4-faa77beeab16 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:10.877300 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6992eda-df05-488e-83f4-faa77beeab16 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:10.877833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6992eda-df05-488e-83f4-faa77beeab16 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:10.878214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6992eda-df05-488e-83f4-faa77beeab16 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:10.878511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6992eda-df05-488e-83f4-faa77beeab16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:10.879867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6992eda-df05-488e-83f4-faa77beeab16 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:10.884767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c2a12-d16b-4e85-a8ab-e240c3b8cc70 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:10.888164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 495/979] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Aug 30 01:04:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:10.891327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6992eda-df05-488e-83f4-faa77beeab16 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:10.929382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbfa9c-fffb-49ab-92fb-5e36298d08d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:10.930018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbfa9c-fffb-49ab-92fb-5e36298d08d1 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:10.930350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbfa9c-fffb-49ab-92fb-5e36298d08d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:10.930626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbfa9c-fffb-49ab-92fb-5e36298d08d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:10.940005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbfa9c-fffb-49ab-92fb-5e36298d08d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:10.941645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbfa9c-fffb-49ab-92fb-5e36298d08d1 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-f5fbfa9c-fffb-49ab-92fb-5e36298d08d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:04:10.954586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:10.955555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:10.956826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:10.956826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:10.956826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:04:10.956826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:04:10.956826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:04:10.956826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:04:10.956826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:04:10.956826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:04:10.956826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:04:10.956826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:10.956826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 485/980] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Aug 30 01:04:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:10.961950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbfa9c-fffb-49ab-92fb-5e36298d08d1 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:11.004968 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6992eda-df05-488e-83f4-faa77beeab16 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:11.050970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d93ea-c1f1-4f9d-b861-a19ee0fe4435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:11.052480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d93ea-c1f1-4f9d-b861-a19ee0fe4435 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:11.052946 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d93ea-c1f1-4f9d-b861-a19ee0fe4435 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:11.053411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d93ea-c1f1-4f9d-b861-a19ee0fe4435 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:11.054770 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d93ea-c1f1-4f9d-b861-a19ee0fe4435 None admin] RBAC: Adding query filter params (interface=admin, service_id=a8272e2f35444ffa9a6fbe8a9acf4735, region_id=RegionOne) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:11.055281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d93ea-c1f1-4f9d-b861-a19ee0fe4435 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:11.057551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d93ea-c1f1-4f9d-b861-a19ee0fe4435 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:11.066005 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 496/981] 10.222.0.44 () {56 vars in 1249 bytes} [Tue Aug 30 01:04:10 2022] GET /identity/v3/endpoints?service_id=a8272e2f35444ffa9a6fbe8a9acf4735&interface=admin®ion_id=RegionOne => generated 193 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:11.071376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d93ea-c1f1-4f9d-b861-a19ee0fe4435 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:11.085437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbfa9c-fffb-49ab-92fb-5e36298d08d1 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:11.106024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:11.110301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:11.130204 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2c2ba-bf7e-496c-98f4-4614b6a1069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:11.130821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2c2ba-bf7e-496c-98f4-4614b6a1069b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:11.131424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2c2ba-bf7e-496c-98f4-4614b6a1069b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:11.131424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2c2ba-bf7e-496c-98f4-4614b6a1069b None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:11.132121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2c2ba-bf7e-496c-98f4-4614b6a1069b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:11.132429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2c2ba-bf7e-496c-98f4-4614b6a1069b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:11.133868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2c2ba-bf7e-496c-98f4-4614b6a1069b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:11.158054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 486/982] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Aug 30 01:04:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:11.163757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2c2ba-bf7e-496c-98f4-4614b6a1069b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:11.213736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d93ea-c1f1-4f9d-b861-a19ee0fe4435 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:11.264231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af4a5d-c673-46f6-929f-af91145b6147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:11.265480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af4a5d-c673-46f6-929f-af91145b6147 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:11.265480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af4a5d-c673-46f6-929f-af91145b6147 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:11.265480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af4a5d-c673-46f6-929f-af91145b6147 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:11.266006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af4a5d-c673-46f6-929f-af91145b6147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:11.267773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af4a5d-c673-46f6-929f-af91145b6147 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:11.288036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 497/983] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Aug 30 01:04:11 2022] GET /identity/v3/services => generated 1727 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:04:11.319654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2c2ba-bf7e-496c-98f4-4614b6a1069b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:11.366950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52463552-b600-4ed6-a36f-d41cb4fb6784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:11.367639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52463552-b600-4ed6-a36f-d41cb4fb6784 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:11.367934 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52463552-b600-4ed6-a36f-d41cb4fb6784 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:11.368230 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52463552-b600-4ed6-a36f-d41cb4fb6784 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:11.430552 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:11.471983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52463552-b600-4ed6-a36f-d41cb4fb6784 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:11.474681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52463552-b600-4ed6-a36f-d41cb4fb6784 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:11.611477 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 487/984] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:04:11 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 01:04:11.618379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af4a5d-c673-46f6-929f-af91145b6147 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:11.756681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:11.771744 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af4a5d-c673-46f6-929f-af91145b6147 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:11.812498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da327c4-7608-49cf-9887-46acdb85bc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:11.813147 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da327c4-7608-49cf-9887-46acdb85bc5d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:11.813478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da327c4-7608-49cf-9887-46acdb85bc5d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:11.813802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da327c4-7608-49cf-9887-46acdb85bc5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:11.826770 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da327c4-7608-49cf-9887-46acdb85bc5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:11.828274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da327c4-7608-49cf-9887-46acdb85bc5d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-1da327c4-7608-49cf-9887-46acdb85bc5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:11.844984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:11.846469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:11.847847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:11.847847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:11.847847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:11.847847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 498/985] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:11.852140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52463552-b600-4ed6-a36f-d41cb4fb6784 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:11.856959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f228f75-a1ed-4431-8d56-e64a87ffe26b None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:11.857400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f228f75-a1ed-4431-8d56-e64a87ffe26b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:11.857509 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f228f75-a1ed-4431-8d56-e64a87ffe26b None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:11.859136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 499/986] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:11.864284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d06794-1389-4871-92c8-f12dfb8f03c7 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:11.864486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d06794-1389-4871-92c8-f12dfb8f03c7 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:11.864752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d06794-1389-4871-92c8-f12dfb8f03c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:11.886868 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b0d06794-1389-4871-92c8-f12dfb8f03c7 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:11.888416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b0d06794-1389-4871-92c8-f12dfb8f03c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:11.978254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:11.982813 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 500/987] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:12.008854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52463552-b600-4ed6-a36f-d41cb4fb6784 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:12.046854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f8682-ade4-4ec2-b699-8c6dfffb4386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:12.047764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f8682-ade4-4ec2-b699-8c6dfffb4386 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:12.048282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f8682-ade4-4ec2-b699-8c6dfffb4386 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:12.048749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f8682-ade4-4ec2-b699-8c6dfffb4386 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:12.049569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f8682-ade4-4ec2-b699-8c6dfffb4386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:12.050123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f8682-ade4-4ec2-b699-8c6dfffb4386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:12.051653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f8682-ade4-4ec2-b699-8c6dfffb4386 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:12.064249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 488/988] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:12.070299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d06794-1389-4871-92c8-f12dfb8f03c7 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:12.143915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:12.176555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d06794-1389-4871-92c8-f12dfb8f03c7 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:12.215243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620be45f-bd96-4192-8ba7-99c13d785248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:12.215905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620be45f-bd96-4192-8ba7-99c13d785248 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:12.216208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620be45f-bd96-4192-8ba7-99c13d785248 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:12.216552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620be45f-bd96-4192-8ba7-99c13d785248 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:12.224093 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620be45f-bd96-4192-8ba7-99c13d785248 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:12.225548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620be45f-bd96-4192-8ba7-99c13d785248 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-620be45f-bd96-4192-8ba7-99c13d785248 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:04:12.237627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:12.238555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:12.239533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:04:12.239533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:04:12.239533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:04:12.239533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:04:12.239533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:04:12.239533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:04:12.239533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:12.239533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:12.243455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 501/989] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:04:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:12.244880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f8682-ade4-4ec2-b699-8c6dfffb4386 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:12.253202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620be45f-bd96-4192-8ba7-99c13d785248 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:12.358486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f8682-ade4-4ec2-b699-8c6dfffb4386 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:12.374123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620be45f-bd96-4192-8ba7-99c13d785248 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:12.393894 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fae6fd-ac0a-4cd9-8e95-842f27501100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:12.394556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fae6fd-ac0a-4cd9-8e95-842f27501100 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:12.394782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fae6fd-ac0a-4cd9-8e95-842f27501100 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:12.395062 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fae6fd-ac0a-4cd9-8e95-842f27501100 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:12.395778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fae6fd-ac0a-4cd9-8e95-842f27501100 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:12.396113 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fae6fd-ac0a-4cd9-8e95-842f27501100 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:12.397539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fae6fd-ac0a-4cd9-8e95-842f27501100 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:12.399600 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:12.406616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:12.412191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 489/990] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:04:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:12.413677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0b613-61e3-4b02-b14b-605261eb64a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:12.414317 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0b613-61e3-4b02-b14b-605261eb64a7 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:12.414595 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0b613-61e3-4b02-b14b-605261eb64a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:12.414925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0b613-61e3-4b02-b14b-605261eb64a7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:12.415551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0b613-61e3-4b02-b14b-605261eb64a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:12.416971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0b613-61e3-4b02-b14b-605261eb64a7 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:12.419256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fae6fd-ac0a-4cd9-8e95-842f27501100 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-a5e0b613-61e3-4b02-b14b-605261eb64a7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:12.428609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:12.429617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:12.430646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 502/991] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:04:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:12.436125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0b613-61e3-4b02-b14b-605261eb64a7 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:12.539655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fae6fd-ac0a-4cd9-8e95-842f27501100 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:12.550400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0b613-61e3-4b02-b14b-605261eb64a7 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:12.577495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bec2bb-f616-4a3f-b0db-b9f8f3d4a579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:12.578101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bec2bb-f616-4a3f-b0db-b9f8f3d4a579 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:12.578385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bec2bb-f616-4a3f-b0db-b9f8f3d4a579 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:12.578652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bec2bb-f616-4a3f-b0db-b9f8f3d4a579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:12.579386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bec2bb-f616-4a3f-b0db-b9f8f3d4a579 None admin] RBAC: Adding query filter params (role.id=71b1513afb19498a9ac3d5584d85750a, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=8a3c3027b61d4f75a67e0a1bb8b52903) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:12.579694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bec2bb-f616-4a3f-b0db-b9f8f3d4a579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:12.581153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bec2bb-f616-4a3f-b0db-b9f8f3d4a579 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:12.587444 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 490/992] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:04:12 2022] GET /identity/v3/role_assignments?user.id=8a3c3027b61d4f75a67e0a1bb8b52903&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=71b1513afb19498a9ac3d5584d85750a => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:12.595956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512593a0-7f16-454f-b3d8-4a5461b6aa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:12.596636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512593a0-7f16-454f-b3d8-4a5461b6aa5f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:12.596916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512593a0-7f16-454f-b3d8-4a5461b6aa5f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:12.597190 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512593a0-7f16-454f-b3d8-4a5461b6aa5f None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:12.597856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512593a0-7f16-454f-b3d8-4a5461b6aa5f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:12.598164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512593a0-7f16-454f-b3d8-4a5461b6aa5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:12.599763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512593a0-7f16-454f-b3d8-4a5461b6aa5f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:12.606490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 503/993] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:04:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:12.612613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bec2bb-f616-4a3f-b0db-b9f8f3d4a579 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:12.692026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:12.740917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bec2bb-f616-4a3f-b0db-b9f8f3d4a579 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:12.777273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce4676-4279-488e-97d6-bfd5fa90900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:12.778057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce4676-4279-488e-97d6-bfd5fa90900e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:12.778344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce4676-4279-488e-97d6-bfd5fa90900e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:12.778641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce4676-4279-488e-97d6-bfd5fa90900e None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:12.779256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce4676-4279-488e-97d6-bfd5fa90900e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:12.779566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce4676-4279-488e-97d6-bfd5fa90900e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:12.780960 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce4676-4279-488e-97d6-bfd5fa90900e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:12.786392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 491/994] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:04:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:12.792238 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512593a0-7f16-454f-b3d8-4a5461b6aa5f None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:12.856030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:12.900386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512593a0-7f16-454f-b3d8-4a5461b6aa5f None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:12.937078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fdd2e-1249-44da-8275-d3846b9fc245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:12.937703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fdd2e-1249-44da-8275-d3846b9fc245 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:12.937985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fdd2e-1249-44da-8275-d3846b9fc245 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:12.938259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fdd2e-1249-44da-8275-d3846b9fc245 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:12.938976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fdd2e-1249-44da-8275-d3846b9fc245 None admin] RBAC: Adding query filter params (interface=public, service_id=eb8033fcb6f84b579014395a150420f2, region_id=RegionOne) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:12.939285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fdd2e-1249-44da-8275-d3846b9fc245 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:12.940718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fdd2e-1249-44da-8275-d3846b9fc245 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:12.947902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 504/995] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Aug 30 01:04:12 2022] GET /identity/v3/endpoints?service_id=eb8033fcb6f84b579014395a150420f2&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:12.955339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce4676-4279-488e-97d6-bfd5fa90900e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:13.087755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce4676-4279-488e-97d6-bfd5fa90900e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:13.112550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:13.127473 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358b0b6-845f-484a-a383-707614da628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:13.128109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358b0b6-845f-484a-a383-707614da628c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:13.128388 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358b0b6-845f-484a-a383-707614da628c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:13.128655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358b0b6-845f-484a-a383-707614da628c None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:13.129256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358b0b6-845f-484a-a383-707614da628c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:13.130609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358b0b6-845f-484a-a383-707614da628c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:13.137327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 492/996] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Aug 30 01:04:12 2022] GET /identity/v3/services => generated 1727 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:04:13.564978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc24a49-e0d3-42f2-a529-078b52c1a7ff None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:13.565510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc24a49-e0d3-42f2-a529-078b52c1a7ff None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:13.565510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc24a49-e0d3-42f2-a529-078b52c1a7ff None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:13.567044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 505/997] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:13.572250 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe549a8e-6c2e-41c8-a2c7-8a0f44fe0fdc None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:13.572541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe549a8e-6c2e-41c8-a2c7-8a0f44fe0fdc None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:13.572815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe549a8e-6c2e-41c8-a2c7-8a0f44fe0fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:13.591768 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-fe549a8e-6c2e-41c8-a2c7-8a0f44fe0fdc None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:13.593000 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-fe549a8e-6c2e-41c8-a2c7-8a0f44fe0fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:13.670051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 493/998] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:13.939693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc24a49-e0d3-42f2-a529-078b52c1a7ff None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:14.048447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc24a49-e0d3-42f2-a529-078b52c1a7ff None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:14.071915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:14.089765 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07afc7-4ca9-403b-8b62-6742e64273e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:14.090700 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07afc7-4ca9-403b-8b62-6742e64273e1 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:14.091085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07afc7-4ca9-403b-8b62-6742e64273e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:14.091535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07afc7-4ca9-403b-8b62-6742e64273e1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:14.092340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07afc7-4ca9-403b-8b62-6742e64273e1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:14.094207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07afc7-4ca9-403b-8b62-6742e64273e1 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-be07afc7-4ca9-403b-8b62-6742e64273e1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:14.103778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:14.104981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:04:14.105882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:14.107488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 506/999] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Aug 30 01:04:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:14.110951 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe549a8e-6c2e-41c8-a2c7-8a0f44fe0fdc None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:14.219287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe549a8e-6c2e-41c8-a2c7-8a0f44fe0fdc None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:14.237638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:14.257551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13081d-0af7-4f0d-a390-320b718a6b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:14.258190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13081d-0af7-4f0d-a390-320b718a6b58 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:14.258471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13081d-0af7-4f0d-a390-320b718a6b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:14.258744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13081d-0af7-4f0d-a390-320b718a6b58 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:14.259400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13081d-0af7-4f0d-a390-320b718a6b58 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:14.259707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13081d-0af7-4f0d-a390-320b718a6b58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:14.261675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13081d-0af7-4f0d-a390-320b718a6b58 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:14.271924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 494/1000] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Aug 30 01:04:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:14.273101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07afc7-4ca9-403b-8b62-6742e64273e1 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:14.377258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07afc7-4ca9-403b-8b62-6742e64273e1 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:14.414460 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dade811-a3bb-4a94-b6c5-f2d5cea25517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:14.415067 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dade811-a3bb-4a94-b6c5-f2d5cea25517 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:14.415342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dade811-a3bb-4a94-b6c5-f2d5cea25517 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:14.415611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dade811-a3bb-4a94-b6c5-f2d5cea25517 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:14.416273 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dade811-a3bb-4a94-b6c5-f2d5cea25517 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:14.416579 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dade811-a3bb-4a94-b6c5-f2d5cea25517 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:14.418255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dade811-a3bb-4a94-b6c5-f2d5cea25517 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:14.428057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 507/1001] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Aug 30 01:04:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:14.429271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13081d-0af7-4f0d-a390-320b718a6b58 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:14.532422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13081d-0af7-4f0d-a390-320b718a6b58 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:14.568046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4663a-5a3e-4089-89ad-c26ac0cf960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:14.568637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4663a-5a3e-4089-89ad-c26ac0cf960d None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:14.568922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4663a-5a3e-4089-89ad-c26ac0cf960d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:14.569190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4663a-5a3e-4089-89ad-c26ac0cf960d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:14.569868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4663a-5a3e-4089-89ad-c26ac0cf960d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:14.571235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4663a-5a3e-4089-89ad-c26ac0cf960d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:14.594615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 495/1002] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:04:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:04:14.640468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37ea23-d113-41c5-a493-501f4784cbc7 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:14.641044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37ea23-d113-41c5-a493-501f4784cbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:14.641044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37ea23-d113-41c5-a493-501f4784cbc7 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:14.642746 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 508/1003] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:14.648766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986660fe-0db0-43b8-b72f-41fd022545e4 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:14.649079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986660fe-0db0-43b8-b72f-41fd022545e4 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:14.649337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986660fe-0db0-43b8-b72f-41fd022545e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:14.669985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-986660fe-0db0-43b8-b72f-41fd022545e4 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:14.671813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-986660fe-0db0-43b8-b72f-41fd022545e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:14.729123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:14.777269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 496/1004] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:15.049677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37ea23-d113-41c5-a493-501f4784cbc7 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:15.110385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:15.130117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8d88f-184e-4364-a8d6-4fec0673c742 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:15.130688 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8d88f-184e-4364-a8d6-4fec0673c742 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:15.130782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8d88f-184e-4364-a8d6-4fec0673c742 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:15.132369 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 497/1005] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:15.138108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae7a4b-b8b0-41f1-806f-f32a0fcef617 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:15.138395 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae7a4b-b8b0-41f1-806f-f32a0fcef617 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:15.138672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae7a4b-b8b0-41f1-806f-f32a0fcef617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:15.157073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-ddae7a4b-b8b0-41f1-806f-f32a0fcef617 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:15.158262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ddae7a4b-b8b0-41f1-806f-f32a0fcef617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:15.169104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37ea23-d113-41c5-a493-501f4784cbc7 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:15.204097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8317b5f-438d-41f0-8f80-78ddcfe76d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:15.204744 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8317b5f-438d-41f0-8f80-78ddcfe76d98 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:15.205023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8317b5f-438d-41f0-8f80-78ddcfe76d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:15.205292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8317b5f-438d-41f0-8f80-78ddcfe76d98 None admin] PATH_INFO: `/v3/services/image` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:15.206014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8317b5f-438d-41f0-8f80-78ddcfe76d98 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:15.207310 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8317b5f-438d-41f0-8f80-78ddcfe76d98 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-b8317b5f-438d-41f0-8f80-78ddcfe76d98 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:15.219939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:15.221924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:15.222942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 509/1006] 10.222.0.44 () {56 vars in 1105 bytes} [Tue Aug 30 01:04:15 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:15.227030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8317b5f-438d-41f0-8f80-78ddcfe76d98 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:15.252503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 498/1007] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:15.335636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8317b5f-438d-41f0-8f80-78ddcfe76d98 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:15.349814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:15.371192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc24e7a-7935-425c-83c8-cf39ed5049e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:15.371809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc24e7a-7935-425c-83c8-cf39ed5049e5 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:15.372110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc24e7a-7935-425c-83c8-cf39ed5049e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:15.372383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc24e7a-7935-425c-83c8-cf39ed5049e5 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:15.373013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc24e7a-7935-425c-83c8-cf39ed5049e5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:15.373315 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc24e7a-7935-425c-83c8-cf39ed5049e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:15.374693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc24e7a-7935-425c-83c8-cf39ed5049e5 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:15.379653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 510/1008] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:15.385124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae7a4b-b8b0-41f1-806f-f32a0fcef617 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:15.482680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:15.489867 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae7a4b-b8b0-41f1-806f-f32a0fcef617 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:15.524248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d292a-1b5a-42ce-958a-de3a205264ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:15.524887 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d292a-1b5a-42ce-958a-de3a205264ce None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:15.525165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d292a-1b5a-42ce-958a-de3a205264ce None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:15.525438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d292a-1b5a-42ce-958a-de3a205264ce None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:15.526067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d292a-1b5a-42ce-958a-de3a205264ce None admin] RBAC: Adding query filter params (type=image) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:15.526372 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d292a-1b5a-42ce-958a-de3a205264ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:15.528668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d292a-1b5a-42ce-958a-de3a205264ce None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:15.533397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 499/1009] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:15 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:15.533707 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc24e7a-7935-425c-83c8-cf39ed5049e5 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:15.649810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc24e7a-7935-425c-83c8-cf39ed5049e5 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:15.690826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d411e38-b82c-4be4-866c-f2223e6ae9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:15.691475 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d411e38-b82c-4be4-866c-f2223e6ae9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:15.691753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d411e38-b82c-4be4-866c-f2223e6ae9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:15.692101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d411e38-b82c-4be4-866c-f2223e6ae9a9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:15.692633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d411e38-b82c-4be4-866c-f2223e6ae9a9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:15.694015 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d411e38-b82c-4be4-866c-f2223e6ae9a9 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-2d411e38-b82c-4be4-866c-f2223e6ae9a9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:15.704544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:15.705811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:15.707390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 511/1010] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:04:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:15.713560 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d292a-1b5a-42ce-958a-de3a205264ce None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:15.818870 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d292a-1b5a-42ce-958a-de3a205264ce None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:15.854510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1576f97-96a9-40a2-9c01-83769580c585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:15.855676 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1576f97-96a9-40a2-9c01-83769580c585 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:15.856223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1576f97-96a9-40a2-9c01-83769580c585 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:15.856223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1576f97-96a9-40a2-9c01-83769580c585 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:15.856864 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1576f97-96a9-40a2-9c01-83769580c585 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:15.857167 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1576f97-96a9-40a2-9c01-83769580c585 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:15.858527 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1576f97-96a9-40a2-9c01-83769580c585 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:15.868041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 500/1011] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:04:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:15.869555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d411e38-b82c-4be4-866c-f2223e6ae9a9 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:15.913756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:15.973246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d411e38-b82c-4be4-866c-f2223e6ae9a9 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:16.009422 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd0755-13ee-462c-9e09-39763fae36de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:16.010031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd0755-13ee-462c-9e09-39763fae36de None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:16.010309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd0755-13ee-462c-9e09-39763fae36de None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:16.010582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd0755-13ee-462c-9e09-39763fae36de None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:16.011296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd0755-13ee-462c-9e09-39763fae36de None admin] RBAC: Adding query filter params (type=compute) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:16.011616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd0755-13ee-462c-9e09-39763fae36de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:16.013044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd0755-13ee-462c-9e09-39763fae36de None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:16.022606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 512/1012] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Aug 30 01:04:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:16.024121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1576f97-96a9-40a2-9c01-83769580c585 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:16.051009 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:16.131007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1576f97-96a9-40a2-9c01-83769580c585 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:16.171809 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee7953-5496-49f4-b5b3-ef6e48af2618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:16.172386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7953-5496-49f4-b5b3-ef6e48af2618 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:16.172669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7953-5496-49f4-b5b3-ef6e48af2618 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:16.172938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee7953-5496-49f4-b5b3-ef6e48af2618 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:16.173625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee7953-5496-49f4-b5b3-ef6e48af2618 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:16.175024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee7953-5496-49f4-b5b3-ef6e48af2618 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:16.197609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 501/1013] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:04:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:04:16.570791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84da00-0c7f-44ad-80f5-79bbbd98d1bc None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:16.571128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84da00-0c7f-44ad-80f5-79bbbd98d1bc None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:16.571348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84da00-0c7f-44ad-80f5-79bbbd98d1bc None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:16.572896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 513/1014] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:16.578755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ab5b3-8bf0-4361-a719-28d0a0fee0da None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:16.579054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ab5b3-8bf0-4361-a719-28d0a0fee0da None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:16.579306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ab5b3-8bf0-4361-a719-28d0a0fee0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:16.600143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-080ab5b3-8bf0-4361-a719-28d0a0fee0da None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:16.601336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-080ab5b3-8bf0-4361-a719-28d0a0fee0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:16.682623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 502/1015] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:16 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:16.956923 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84da00-0c7f-44ad-80f5-79bbbd98d1bc None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:17.077263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84da00-0c7f-44ad-80f5-79bbbd98d1bc None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:17.086219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:17.118052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05d134-b86d-463d-9f72-e92c9193b42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:17.118684 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05d134-b86d-463d-9f72-e92c9193b42b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:17.118955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05d134-b86d-463d-9f72-e92c9193b42b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:17.119221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05d134-b86d-463d-9f72-e92c9193b42b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:17.120723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05d134-b86d-463d-9f72-e92c9193b42b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:17.122172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05d134-b86d-463d-9f72-e92c9193b42b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:17.124051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 514/1016] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:17.131202 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ab5b3-8bf0-4361-a719-28d0a0fee0da None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:17.140648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:17.254718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ab5b3-8bf0-4361-a719-28d0a0fee0da None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:17.296681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b787a6-8567-475b-910e-9b124ed1a4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:17.297660 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b787a6-8567-475b-910e-9b124ed1a4b5 None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:17.298129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b787a6-8567-475b-910e-9b124ed1a4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:17.298584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b787a6-8567-475b-910e-9b124ed1a4b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:17.299767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b787a6-8567-475b-910e-9b124ed1a4b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:17.302357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b787a6-8567-475b-910e-9b124ed1a4b5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:17.348549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 503/1017] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:04:17 2022] POST /identity/v3/projects => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:04:17.551245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965e60c-0bb0-4020-a544-a2ef15d0da45 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:17.551598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965e60c-0bb0-4020-a544-a2ef15d0da45 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:17.552101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965e60c-0bb0-4020-a544-a2ef15d0da45 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:17.554974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 515/1018] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:17.562272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8965ff-9be4-4cd5-8f26-006cdff25e1b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:17.562718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8965ff-9be4-4cd5-8f26-006cdff25e1b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:17.563128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8965ff-9be4-4cd5-8f26-006cdff25e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:17.588334 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-ef8965ff-9be4-4cd5-8f26-006cdff25e1b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:17.590426 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ef8965ff-9be4-4cd5-8f26-006cdff25e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:17.647284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:17.664952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 504/1019] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:17 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:17.934280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965e60c-0bb0-4020-a544-a2ef15d0da45 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:18.067628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965e60c-0bb0-4020-a544-a2ef15d0da45 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:18.101558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1c994-42e3-4a37-b765-f373e802c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:18.102164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1c994-42e3-4a37-b765-f373e802c0a4 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:18.102437 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1c994-42e3-4a37-b765-f373e802c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:18.102701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1c994-42e3-4a37-b765-f373e802c0a4 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:18.103359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1c994-42e3-4a37-b765-f373e802c0a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:18.105384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1c994-42e3-4a37-b765-f373e802c0a4 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:18.120030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 516/1020] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:04:17 2022] POST /identity/v3/services => generated 242 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:04:18.306805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f547bed-ecd5-4701-a8e9-d3f70cb8940b None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:18.307327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f547bed-ecd5-4701-a8e9-d3f70cb8940b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:18.307327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f547bed-ecd5-4701-a8e9-d3f70cb8940b None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:18.309519 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 505/1021] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:18.314916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a40a0-52bb-4cd2-9210-76d233e886ff None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:18.315202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a40a0-52bb-4cd2-9210-76d233e886ff None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:18.315477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a40a0-52bb-4cd2-9210-76d233e886ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:18.345383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-667a40a0-52bb-4cd2-9210-76d233e886ff None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:18.346741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-667a40a0-52bb-4cd2-9210-76d233e886ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:18.381950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:18.437323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 517/1022] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:18.716804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f547bed-ecd5-4701-a8e9-d3f70cb8940b None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:18.827822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f547bed-ecd5-4701-a8e9-d3f70cb8940b None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:18.864008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09d24d-2f91-4f20-8014-e7653921c7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:18.864811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09d24d-2f91-4f20-8014-e7653921c7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:18.865090 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09d24d-2f91-4f20-8014-e7653921c7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:18.865358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09d24d-2f91-4f20-8014-e7653921c7a5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:18.876423 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09d24d-2f91-4f20-8014-e7653921c7a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-da09d24d-2f91-4f20-8014-e7653921c7a5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:04:18.881242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:18.882290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:18.883333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:18.883333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:18.883333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:18.883333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:18.883333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:04:18.883333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:04:18.883333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:04:18.883333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:04:18.883333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:04:18.883333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:04:18.883333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:04:18.883333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:18.888075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 506/1023] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Aug 30 01:04:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:18.889542 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a40a0-52bb-4cd2-9210-76d233e886ff None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:18.998518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a40a0-52bb-4cd2-9210-76d233e886ff None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:19.033588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef67456-9945-4348-ac3b-76ea12d73e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:19.034195 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef67456-9945-4348-ac3b-76ea12d73e65 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:19.034468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef67456-9945-4348-ac3b-76ea12d73e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:19.034734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef67456-9945-4348-ac3b-76ea12d73e65 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:19.035402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef67456-9945-4348-ac3b-76ea12d73e65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:19.035696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef67456-9945-4348-ac3b-76ea12d73e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:19.037087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef67456-9945-4348-ac3b-76ea12d73e65 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:19.051056 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 518/1024] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Aug 30 01:04:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:19.054159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09d24d-2f91-4f20-8014-e7653921c7a5 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:19.109242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:19.164284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09d24d-2f91-4f20-8014-e7653921c7a5 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:19.199635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac81f1-42a5-4626-8f14-9cba035b9e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:19.200298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac81f1-42a5-4626-8f14-9cba035b9e59 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:19.200574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac81f1-42a5-4626-8f14-9cba035b9e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:19.200836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac81f1-42a5-4626-8f14-9cba035b9e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:19.211706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac81f1-42a5-4626-8f14-9cba035b9e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:19.213186 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac81f1-42a5-4626-8f14-9cba035b9e59 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-4eac81f1-42a5-4626-8f14-9cba035b9e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:19.229183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:19.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:19.231778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:19.231778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:19.231778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:19.231778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 507/1025] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:19.238314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef67456-9945-4348-ac3b-76ea12d73e65 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:19.266904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:19.348333 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef67456-9945-4348-ac3b-76ea12d73e65 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:19.381206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea916a7a-32c5-4c91-b90d-42117b1846d1 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:19.381740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea916a7a-32c5-4c91-b90d-42117b1846d1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:19.381740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea916a7a-32c5-4c91-b90d-42117b1846d1 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:19.383177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 508/1026] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:19.383290 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114412e6-f6bf-4d6a-8cac-c44d362aca80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:19.383767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114412e6-f6bf-4d6a-8cac-c44d362aca80 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:19.384068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114412e6-f6bf-4d6a-8cac-c44d362aca80 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:19.384338 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114412e6-f6bf-4d6a-8cac-c44d362aca80 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:19.384989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114412e6-f6bf-4d6a-8cac-c44d362aca80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:19.385286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114412e6-f6bf-4d6a-8cac-c44d362aca80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:19.387068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114412e6-f6bf-4d6a-8cac-c44d362aca80 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:19.388028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99225617-20c4-4c45-a5a8-94456696c89a None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:19.388306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99225617-20c4-4c45-a5a8-94456696c89a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:19.388576 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99225617-20c4-4c45-a5a8-94456696c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:19.403254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 519/1027] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:19.407827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-99225617-20c4-4c45-a5a8-94456696c89a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:19.409038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114412e6-f6bf-4d6a-8cac-c44d362aca80 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:19.409181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-99225617-20c4-4c45-a5a8-94456696c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:19.488566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 509/1028] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:19.502892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:19.524085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114412e6-f6bf-4d6a-8cac-c44d362aca80 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:19.560494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701293d-455a-4320-b18f-d59ee04c6973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:19.561125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701293d-455a-4320-b18f-d59ee04c6973 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:19.561395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701293d-455a-4320-b18f-d59ee04c6973 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:19.561828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701293d-455a-4320-b18f-d59ee04c6973 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:19.569523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701293d-455a-4320-b18f-d59ee04c6973 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:19.570978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701293d-455a-4320-b18f-d59ee04c6973 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-0701293d-455a-4320-b18f-d59ee04c6973 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:04:19.583704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:19.584787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:19.590902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:19.590902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:19.590902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:04:19.590902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:04:19.590902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:04:19.590902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:04:19.590902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:04:19.590902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:04:19.590902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:04:19.590902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:19.590902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 520/1029] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Aug 30 01:04:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:19.591546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99225617-20c4-4c45-a5a8-94456696c89a None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:19.624116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:19.703214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99225617-20c4-4c45-a5a8-94456696c89a None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:19.739310 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218754c5-66f5-48b7-8a54-3aff38a8a20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:19.739922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218754c5-66f5-48b7-8a54-3aff38a8a20c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:19.740293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218754c5-66f5-48b7-8a54-3aff38a8a20c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:19.740569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218754c5-66f5-48b7-8a54-3aff38a8a20c None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:19.741294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218754c5-66f5-48b7-8a54-3aff38a8a20c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:19.741596 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218754c5-66f5-48b7-8a54-3aff38a8a20c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:19.743005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218754c5-66f5-48b7-8a54-3aff38a8a20c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:19.758412 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701293d-455a-4320-b18f-d59ee04c6973 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:19.766788 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 510/1030] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Aug 30 01:04:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:19.767897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218754c5-66f5-48b7-8a54-3aff38a8a20c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:19.864846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:19.873892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701293d-455a-4320-b18f-d59ee04c6973 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:19.878822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218754c5-66f5-48b7-8a54-3aff38a8a20c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:19.916952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdcc6a-e7dd-4ea0-8aa3-30979b219cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:19.917934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdcc6a-e7dd-4ea0-8aa3-30979b219cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:19.918266 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdcc6a-e7dd-4ea0-8aa3-30979b219cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:19.918506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdcc6a-e7dd-4ea0-8aa3-30979b219cb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:19.919618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdcc6a-e7dd-4ea0-8aa3-30979b219cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:19.920648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b373b04-0f4b-47e9-aa52-a934b9aefc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:19.921103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdcc6a-e7dd-4ea0-8aa3-30979b219cb6 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:19.921219 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b373b04-0f4b-47e9-aa52-a934b9aefc49 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:19.921458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b373b04-0f4b-47e9-aa52-a934b9aefc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:19.921623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b373b04-0f4b-47e9-aa52-a934b9aefc49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:19.922854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 521/1031] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:19.928726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdcc6a-e7dd-4ea0-8aa3-30979b219cb6 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:19.944442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:20.001430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b373b04-0f4b-47e9-aa52-a934b9aefc49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:20.002885 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b373b04-0f4b-47e9-aa52-a934b9aefc49 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:20.069291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdcc6a-e7dd-4ea0-8aa3-30979b219cb6 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:20.107103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64930f-223c-4209-ab6a-aa73ebaf288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:20.107721 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64930f-223c-4209-ab6a-aa73ebaf288c None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:20.108004 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64930f-223c-4209-ab6a-aa73ebaf288c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:20.108287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64930f-223c-4209-ab6a-aa73ebaf288c None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:20.108940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64930f-223c-4209-ab6a-aa73ebaf288c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:20.110367 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64930f-223c-4209-ab6a-aa73ebaf288c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:20.136603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:20.137388 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 522/1032] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:04:19 2022] POST /identity/v3/users => generated 285 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:04:20.169256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 511/1033] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:04:19 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:04:20.175291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64930f-223c-4209-ab6a-aa73ebaf288c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:20.191944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607eff9-1941-479a-98fe-02b4a1654499 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:20.192344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607eff9-1941-479a-98fe-02b4a1654499 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:20.192532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607eff9-1941-479a-98fe-02b4a1654499 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:20.194214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 512/1034] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:20.200872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541d3b0-381c-42ff-bbf2-7031c5c2721e None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:20.201209 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541d3b0-381c-42ff-bbf2-7031c5c2721e None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:20.201422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541d3b0-381c-42ff-bbf2-7031c5c2721e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:20.233238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-7541d3b0-381c-42ff-bbf2-7031c5c2721e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:20.235329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7541d3b0-381c-42ff-bbf2-7031c5c2721e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:20.294601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:20.344056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 513/1035] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:20.354884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64930f-223c-4209-ab6a-aa73ebaf288c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:20.407471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77935226-2bb9-4499-a33b-a1b7f37cb31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:20.408462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77935226-2bb9-4499-a33b-a1b7f37cb31a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:20.408949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77935226-2bb9-4499-a33b-a1b7f37cb31a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:20.409434 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77935226-2bb9-4499-a33b-a1b7f37cb31a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:20.429339 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77935226-2bb9-4499-a33b-a1b7f37cb31a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:20.432054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77935226-2bb9-4499-a33b-a1b7f37cb31a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-77935226-2bb9-4499-a33b-a1b7f37cb31a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:20.461308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:20.462401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:20.463287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:20.463287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:20.463287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:20.464115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 523/1036] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:20.470834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541d3b0-381c-42ff-bbf2-7031c5c2721e None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:20.587357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541d3b0-381c-42ff-bbf2-7031c5c2721e None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:20.610147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:20.631005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7171315-ad43-4cab-80ad-9578e447206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:20.631661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7171315-ad43-4cab-80ad-9578e447206d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:20.631940 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7171315-ad43-4cab-80ad-9578e447206d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:20.632240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7171315-ad43-4cab-80ad-9578e447206d None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:20.632890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7171315-ad43-4cab-80ad-9578e447206d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:20.633194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7171315-ad43-4cab-80ad-9578e447206d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:20.634721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7171315-ad43-4cab-80ad-9578e447206d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:20.649111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 514/1037] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:20.656703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77935226-2bb9-4499-a33b-a1b7f37cb31a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:20.676374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7171315-ad43-4cab-80ad-9578e447206d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:20.788460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7171315-ad43-4cab-80ad-9578e447206d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:20.821939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77935226-2bb9-4499-a33b-a1b7f37cb31a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:20.826936 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0b72-6b5b-44f9-b1c1-c7c4735096b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:20.827690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0b72-6b5b-44f9-b1c1-c7c4735096b3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:20.828065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0b72-6b5b-44f9-b1c1-c7c4735096b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:20.828345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0b72-6b5b-44f9-b1c1-c7c4735096b3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:20.828949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c0b72-6b5b-44f9-b1c1-c7c4735096b3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:20.830886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c0b72-6b5b-44f9-b1c1-c7c4735096b3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:20.841920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-596c0b72-6b5b-44f9-b1c1-c7c4735096b3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:20.842257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:20.843377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:20.845142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 515/1038] 10.222.0.44 () {56 vars in 1105 bytes} [Tue Aug 30 01:04:20 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:20.849595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0b72-6b5b-44f9-b1c1-c7c4735096b3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:20.864463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b610b6-d409-460b-bd05-56bc94dd79fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:20.865099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b610b6-d409-460b-bd05-56bc94dd79fe None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:20.865375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b610b6-d409-460b-bd05-56bc94dd79fe None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:20.865642 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b610b6-d409-460b-bd05-56bc94dd79fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:20.873773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b610b6-d409-460b-bd05-56bc94dd79fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:20.875235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b610b6-d409-460b-bd05-56bc94dd79fe None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-30b610b6-d409-460b-bd05-56bc94dd79fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:04:20.887794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:20.888924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:20.889943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:04:20.889943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:04:20.889943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:04:20.889943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:04:20.889943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:04:20.889943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:04:20.889943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:20.889943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:20.889943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 524/1039] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:04:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:20.894937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b610b6-d409-460b-bd05-56bc94dd79fe None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:20.901854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:20.968171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0b72-6b5b-44f9-b1c1-c7c4735096b3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:21.005195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ab12b-4f2e-4b84-8a62-a9ec12e2ae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:21.005801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ab12b-4f2e-4b84-8a62-a9ec12e2ae2e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:21.005932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b610b6-d409-460b-bd05-56bc94dd79fe None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:21.006097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ab12b-4f2e-4b84-8a62-a9ec12e2ae2e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:21.006345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ab12b-4f2e-4b84-8a62-a9ec12e2ae2e None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:21.006977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ab12b-4f2e-4b84-8a62-a9ec12e2ae2e None admin] RBAC: Adding query filter params (name=image) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:21.007286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ab12b-4f2e-4b84-8a62-a9ec12e2ae2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:21.008689 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ab12b-4f2e-4b84-8a62-a9ec12e2ae2e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:21.014798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 516/1040] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:21.020425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ab12b-4f2e-4b84-8a62-a9ec12e2ae2e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:21.042750 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cb480-d526-4a20-ba7b-1d391e0e76fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:21.043626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cb480-d526-4a20-ba7b-1d391e0e76fa None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:21.043741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cb480-d526-4a20-ba7b-1d391e0e76fa None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:21.043987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cb480-d526-4a20-ba7b-1d391e0e76fa None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:21.044740 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cb480-d526-4a20-ba7b-1d391e0e76fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:21.045037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cb480-d526-4a20-ba7b-1d391e0e76fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:21.046474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cb480-d526-4a20-ba7b-1d391e0e76fa None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:21.064426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 525/1041] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:04:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:21.065588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cb480-d526-4a20-ba7b-1d391e0e76fa None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:21.107144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:21.136886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ab12b-4f2e-4b84-8a62-a9ec12e2ae2e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:21.144868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:21.177695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cb480-d526-4a20-ba7b-1d391e0e76fa None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:21.179594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095199a-fefa-414f-bf71-8b534f0fb963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:21.180339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095199a-fefa-414f-bf71-8b534f0fb963 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:21.180684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095199a-fefa-414f-bf71-8b534f0fb963 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:21.181024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095199a-fefa-414f-bf71-8b534f0fb963 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:21.181731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095199a-fefa-414f-bf71-8b534f0fb963 None admin] RBAC: Adding query filter params (type=image) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:21.182100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095199a-fefa-414f-bf71-8b534f0fb963 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:21.183789 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095199a-fefa-414f-bf71-8b534f0fb963 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:21.189469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 517/1042] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:21.195438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095199a-fefa-414f-bf71-8b534f0fb963 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:21.223314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf576b-a943-4052-98db-606f5295c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:21.223847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf576b-a943-4052-98db-606f5295c25f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:21.224171 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf576b-a943-4052-98db-606f5295c25f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:21.224441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf576b-a943-4052-98db-606f5295c25f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:21.225182 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf576b-a943-4052-98db-606f5295c25f None admin] RBAC: Adding query filter params (role.id=71b1513afb19498a9ac3d5584d85750a, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=049399a752ed4646a13f8e4a3a62d8e9) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:21.225490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf576b-a943-4052-98db-606f5295c25f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:21.226932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf576b-a943-4052-98db-606f5295c25f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:21.234092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 526/1043] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:04:21 2022] GET /identity/v3/role_assignments?user.id=049399a752ed4646a13f8e4a3a62d8e9&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=71b1513afb19498a9ac3d5584d85750a => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:21.323321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095199a-fefa-414f-bf71-8b534f0fb963 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:21.364736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032f2c7-652d-4a89-a8e1-a9edf30cce4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:21.365428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032f2c7-652d-4a89-a8e1-a9edf30cce4e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:21.365712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032f2c7-652d-4a89-a8e1-a9edf30cce4e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:21.365982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032f2c7-652d-4a89-a8e1-a9edf30cce4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:21.366734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032f2c7-652d-4a89-a8e1-a9edf30cce4e None admin] RBAC: Adding query filter params (interface=public, service_id=d9d85bc3c11b46b294aacf94c85fda19, region_id=RegionOne) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:21.367045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032f2c7-652d-4a89-a8e1-a9edf30cce4e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:21.368554 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032f2c7-652d-4a89-a8e1-a9edf30cce4e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:21.380859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 518/1044] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Aug 30 01:04:21 2022] GET /identity/v3/endpoints?service_id=d9d85bc3c11b46b294aacf94c85fda19&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:21.382805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf576b-a943-4052-98db-606f5295c25f None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:21.493749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf576b-a943-4052-98db-606f5295c25f None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:21.496116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:21.539672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04023b0a-85eb-4d46-a58f-a81fc1189f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:21.540662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04023b0a-85eb-4d46-a58f-a81fc1189f26 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:21.541095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04023b0a-85eb-4d46-a58f-a81fc1189f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:21.541533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04023b0a-85eb-4d46-a58f-a81fc1189f26 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:21.542761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04023b0a-85eb-4d46-a58f-a81fc1189f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:21.544491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04023b0a-85eb-4d46-a58f-a81fc1189f26 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:21.556206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 527/1045] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Aug 30 01:04:21 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:04:22.167341 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ae81a-d554-4e45-81ab-bece11652281 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:22.167866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ae81a-d554-4e45-81ab-bece11652281 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:22.167866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ae81a-d554-4e45-81ab-bece11652281 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:22.169388 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 519/1046] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:22.174475 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ae929-7212-4f79-9e94-04ac13de85be None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:22.174761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ae929-7212-4f79-9e94-04ac13de85be None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:22.175034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ae929-7212-4f79-9e94-04ac13de85be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:22.194438 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6a3ae929-7212-4f79-9e94-04ac13de85be None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:22.195622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6a3ae929-7212-4f79-9e94-04ac13de85be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:22.270137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 528/1047] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:22.545166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ae81a-d554-4e45-81ab-bece11652281 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:22.585627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:22.663088 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ae81a-d554-4e45-81ab-bece11652281 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:22.704610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728ad35-e938-4766-9a7e-5d348850b3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:22.705249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728ad35-e938-4766-9a7e-5d348850b3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:22.705524 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728ad35-e938-4766-9a7e-5d348850b3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:22.705792 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728ad35-e938-4766-9a7e-5d348850b3e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:22.721495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728ad35-e938-4766-9a7e-5d348850b3e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-a728ad35-e938-4766-9a7e-5d348850b3e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:04:22.726401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:22.727346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:22.729004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 520/1048] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:04:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:22.734129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ae929-7212-4f79-9e94-04ac13de85be None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:22.875197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:22.885053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ae929-7212-4f79-9e94-04ac13de85be None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:22.945804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852b812-b5fb-4f13-98c0-71ff69827045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:22.946808 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852b812-b5fb-4f13-98c0-71ff69827045 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:22.947280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852b812-b5fb-4f13-98c0-71ff69827045 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:22.947728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852b812-b5fb-4f13-98c0-71ff69827045 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:22.949098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852b812-b5fb-4f13-98c0-71ff69827045 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:22.949609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852b812-b5fb-4f13-98c0-71ff69827045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:22.952055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852b812-b5fb-4f13-98c0-71ff69827045 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:22.962693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 529/1049] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:04:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:22.969458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728ad35-e938-4766-9a7e-5d348850b3e3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:23.106457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:23.115529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728ad35-e938-4766-9a7e-5d348850b3e3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:23.160481 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad7d93-4604-46ec-b2be-ec93f202a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:23.161069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad7d93-4604-46ec-b2be-ec93f202a369 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:23.161346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad7d93-4604-46ec-b2be-ec93f202a369 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:23.161619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad7d93-4604-46ec-b2be-ec93f202a369 None admin] PATH_INFO: `/v3/users/7cffb5e6721142cea4990b4be58bff7e` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:23.179602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad7d93-4604-46ec-b2be-ec93f202a369 None admin] RBAC: Authorizing `identity:get_user(user_id=7cffb5e6721142cea4990b4be58bff7e)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:23.181303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad7d93-4604-46ec-b2be-ec93f202a369 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:23.184134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 521/1050] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:04:22 2022] GET /identity/v3/users/7cffb5e6721142cea4990b4be58bff7e => generated 285 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:23.190639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852b812-b5fb-4f13-98c0-71ff69827045 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:23.235137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e18226-c62b-4c01-9da4-c2610967a382 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:23.235415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e18226-c62b-4c01-9da4-c2610967a382 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:23.235683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e18226-c62b-4c01-9da4-c2610967a382 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:23.237103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 522/1051] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:23.242362 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9edd7-b6c5-4a55-a833-b1aa32d4daa2 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:23.242740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9edd7-b6c5-4a55-a833-b1aa32d4daa2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:23.243019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9edd7-b6c5-4a55-a833-b1aa32d4daa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:23.266747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-02c9edd7-b6c5-4a55-a833-b1aa32d4daa2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:23.267980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-02c9edd7-b6c5-4a55-a833-b1aa32d4daa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:23.324441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852b812-b5fb-4f13-98c0-71ff69827045 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:23.346910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 523/1052] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:23.367710 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d269ac-489d-4de0-819b-b685e863079c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:23.368262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d269ac-489d-4de0-819b-b685e863079c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:23.368871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d269ac-489d-4de0-819b-b685e863079c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:23.368871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d269ac-489d-4de0-819b-b685e863079c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:23.400446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:23.445841 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d269ac-489d-4de0-819b-b685e863079c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:23.447308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d269ac-489d-4de0-819b-b685e863079c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:23.572373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 530/1053] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:04:23 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:04:23.578285 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9edd7-b6c5-4a55-a833-b1aa32d4daa2 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:23.609172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce20f53-92ea-42ba-a8b8-ceb0c102c69a None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:23.609607 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce20f53-92ea-42ba-a8b8-ceb0c102c69a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:23.609989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce20f53-92ea-42ba-a8b8-ceb0c102c69a None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:23.611948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 531/1054] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:23.618525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85885a5a-2653-4589-8aa0-46c6dc283513 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:23.618808 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85885a5a-2653-4589-8aa0-46c6dc283513 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:23.619083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85885a5a-2653-4589-8aa0-46c6dc283513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:23.638562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-85885a5a-2653-4589-8aa0-46c6dc283513 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:23.639752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-85885a5a-2653-4589-8aa0-46c6dc283513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:23.672288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:23.692476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9edd7-b6c5-4a55-a833-b1aa32d4daa2 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:23.696248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:23.712328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 532/1055] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:23.712328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85885a5a-2653-4589-8aa0-46c6dc283513 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:23.731964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313b737-0505-4757-8661-8ae33841599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:23.732895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313b737-0505-4757-8661-8ae33841599d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:23.734324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313b737-0505-4757-8661-8ae33841599d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:23.734324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313b737-0505-4757-8661-8ae33841599d None admin] PATH_INFO: `/v3/projects/3b91df1085544a21ba9adf02b065c040` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:23.734940 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313b737-0505-4757-8661-8ae33841599d None admin] RBAC: Authorizing `identity:get_project(project_id=3b91df1085544a21ba9adf02b065c040)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:23.736495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313b737-0505-4757-8661-8ae33841599d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:23.738298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 524/1056] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:04:23 2022] GET /identity/v3/projects/3b91df1085544a21ba9adf02b065c040 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:23.744281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313b737-0505-4757-8661-8ae33841599d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:23.824434 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85885a5a-2653-4589-8aa0-46c6dc283513 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:23.857773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313b737-0505-4757-8661-8ae33841599d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:23.860181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd4e12-05f6-4189-a419-074531909004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:23.860822 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd4e12-05f6-4189-a419-074531909004 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:23.861128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd4e12-05f6-4189-a419-074531909004 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:23.861396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd4e12-05f6-4189-a419-074531909004 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:23.874872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd4e12-05f6-4189-a419-074531909004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-d4cd4e12-05f6-4189-a419-074531909004 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:04:23.880152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:23.881094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:23.881911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:23.881911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:23.881911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:23.881911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:23.881911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:04:23.881911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:04:23.881911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:04:23.881911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:04:23.881911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:04:23.881911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:04:23.881911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:04:23.881911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:23.884247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 533/1057] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Aug 30 01:04:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:23.891423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd4e12-05f6-4189-a419-074531909004 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:23.895941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c11df-efa9-49f1-8fd8-a3cf5883897d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:23.897452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c11df-efa9-49f1-8fd8-a3cf5883897d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:23.897739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c11df-efa9-49f1-8fd8-a3cf5883897d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:23.898010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c11df-efa9-49f1-8fd8-a3cf5883897d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:23.898746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c11df-efa9-49f1-8fd8-a3cf5883897d None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=3b91df1085544a21ba9adf02b065c040, user.id=7cffb5e6721142cea4990b4be58bff7e) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:23.899055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c11df-efa9-49f1-8fd8-a3cf5883897d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:23.900515 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c11df-efa9-49f1-8fd8-a3cf5883897d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:23.908125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 525/1058] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:04:23 2022] GET /identity/v3/role_assignments?user.id=7cffb5e6721142cea4990b4be58bff7e&scope.project.id=3b91df1085544a21ba9adf02b065c040&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:23.956805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:23.981696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c11df-efa9-49f1-8fd8-a3cf5883897d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:24.000198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:24.023944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd4e12-05f6-4189-a419-074531909004 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:24.070792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba2f2d-7202-4eb1-a306-b799ad027952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:24.071412 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba2f2d-7202-4eb1-a306-b799ad027952 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:24.071688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba2f2d-7202-4eb1-a306-b799ad027952 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:24.071957 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba2f2d-7202-4eb1-a306-b799ad027952 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:24.072669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba2f2d-7202-4eb1-a306-b799ad027952 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:24.073498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba2f2d-7202-4eb1-a306-b799ad027952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:24.074883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba2f2d-7202-4eb1-a306-b799ad027952 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:24.083844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 534/1059] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Aug 30 01:04:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:24.090366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba2f2d-7202-4eb1-a306-b799ad027952 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:24.103117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c11df-efa9-49f1-8fd8-a3cf5883897d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:24.148768 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad76878-3a0a-41f0-8851-8d8e8bc81584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:24.149485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad76878-3a0a-41f0-8851-8d8e8bc81584 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:24.149812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad76878-3a0a-41f0-8851-8d8e8bc81584 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:24.150092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad76878-3a0a-41f0-8851-8d8e8bc81584 None admin] PATH_INFO: `/v3/services/image` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:24.150829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad76878-3a0a-41f0-8851-8d8e8bc81584 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:24.152509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad76878-3a0a-41f0-8851-8d8e8bc81584 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-9ad76878-3a0a-41f0-8851-8d8e8bc81584 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:24.162974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:24.164265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:24.165166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 526/1060] 10.222.0.44 () {56 vars in 1105 bytes} [Tue Aug 30 01:04:23 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:24.171847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad76878-3a0a-41f0-8851-8d8e8bc81584 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:24.225620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba2f2d-7202-4eb1-a306-b799ad027952 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:24.237727 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:24.263891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a27d6c-db46-448a-80d0-f9d93622d529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:24.264675 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a27d6c-db46-448a-80d0-f9d93622d529 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:24.264979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a27d6c-db46-448a-80d0-f9d93622d529 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:24.265247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a27d6c-db46-448a-80d0-f9d93622d529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:24.270362 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:24.280235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a27d6c-db46-448a-80d0-f9d93622d529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:24.282113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a27d6c-db46-448a-80d0-f9d93622d529 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-c6a27d6c-db46-448a-80d0-f9d93622d529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:24.298623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:24.299669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:24.300620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:24.300620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:24.300620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:24.300620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 535/1061] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:24.305081 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad76878-3a0a-41f0-8851-8d8e8bc81584 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:24.305620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a27d6c-db46-448a-80d0-f9d93622d529 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:24.348772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942267c-7170-4e30-b236-f7d9855b12eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:24.349462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942267c-7170-4e30-b236-f7d9855b12eb None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:24.349752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942267c-7170-4e30-b236-f7d9855b12eb None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:24.350022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942267c-7170-4e30-b236-f7d9855b12eb None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:24.350660 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942267c-7170-4e30-b236-f7d9855b12eb None admin] RBAC: Adding query filter params (name=image) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:24.350965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942267c-7170-4e30-b236-f7d9855b12eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:24.352376 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942267c-7170-4e30-b236-f7d9855b12eb None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:24.360032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 527/1062] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:24.364030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942267c-7170-4e30-b236-f7d9855b12eb None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:24.427369 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a27d6c-db46-448a-80d0-f9d93622d529 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:24.469519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aec94-045a-49a4-8f9f-5f536caf4f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:24.470564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aec94-045a-49a4-8f9f-5f536caf4f33 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:24.470564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aec94-045a-49a4-8f9f-5f536caf4f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:24.470753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aec94-045a-49a4-8f9f-5f536caf4f33 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:24.471302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8aec94-045a-49a4-8f9f-5f536caf4f33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:24.471615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8aec94-045a-49a4-8f9f-5f536caf4f33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:24.473024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8aec94-045a-49a4-8f9f-5f536caf4f33 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:24.489269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 536/1063] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:24.489652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942267c-7170-4e30-b236-f7d9855b12eb None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:24.495495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aec94-045a-49a4-8f9f-5f536caf4f33 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:24.521796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:24.545655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf547bf1-9345-4d8a-911c-99c01d49f892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:24.546284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf547bf1-9345-4d8a-911c-99c01d49f892 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:24.546565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf547bf1-9345-4d8a-911c-99c01d49f892 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:24.546904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf547bf1-9345-4d8a-911c-99c01d49f892 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:24.549610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf547bf1-9345-4d8a-911c-99c01d49f892 None admin] RBAC: Adding query filter params (type=image) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:24.549918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf547bf1-9345-4d8a-911c-99c01d49f892 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:24.553727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf547bf1-9345-4d8a-911c-99c01d49f892 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:24.564035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 528/1064] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:24.566289 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf547bf1-9345-4d8a-911c-99c01d49f892 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:24.576465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:24.646014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aec94-045a-49a4-8f9f-5f536caf4f33 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:24.688604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee6a02-5aaf-4ddb-b859-3bef7b9c4933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:24.689251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee6a02-5aaf-4ddb-b859-3bef7b9c4933 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:24.689530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee6a02-5aaf-4ddb-b859-3bef7b9c4933 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:24.689803 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee6a02-5aaf-4ddb-b859-3bef7b9c4933 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:24.698850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee6a02-5aaf-4ddb-b859-3bef7b9c4933 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:24.699502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf547bf1-9345-4d8a-911c-99c01d49f892 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:24.700363 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee6a02-5aaf-4ddb-b859-3bef7b9c4933 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-95ee6a02-5aaf-4ddb-b859-3bef7b9c4933 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:04:24.712907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:24.713905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:24.714826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:24.714826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:24.714826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:04:24.714826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:04:24.714826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:04:24.714826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:04:24.714826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:04:24.714826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:04:24.714826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:04:24.714826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:24.719673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 537/1065] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Aug 30 01:04:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:24.720503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee6a02-5aaf-4ddb-b859-3bef7b9c4933 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:24.756106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd64c1-3b67-4c2d-85d6-dd2f52b2cb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:24.757203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd64c1-3b67-4c2d-85d6-dd2f52b2cb3a None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:24.757649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd64c1-3b67-4c2d-85d6-dd2f52b2cb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:24.758069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd64c1-3b67-4c2d-85d6-dd2f52b2cb3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:24.759075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd64c1-3b67-4c2d-85d6-dd2f52b2cb3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:24.761623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd64c1-3b67-4c2d-85d6-dd2f52b2cb3a None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:24.785819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 529/1066] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:04:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:04:24.818353 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:24.849946 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee6a02-5aaf-4ddb-b859-3bef7b9c4933 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:24.904078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224ac18-a5b0-4980-b75b-c5f636d03c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:24.904078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224ac18-a5b0-4980-b75b-c5f636d03c37 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:24.904078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224ac18-a5b0-4980-b75b-c5f636d03c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:24.904078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224ac18-a5b0-4980-b75b-c5f636d03c37 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:24.904523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224ac18-a5b0-4980-b75b-c5f636d03c37 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:24.904523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224ac18-a5b0-4980-b75b-c5f636d03c37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:24.905243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224ac18-a5b0-4980-b75b-c5f636d03c37 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:24.931544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 538/1067] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Aug 30 01:04:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:24.939012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd64c1-3b67-4c2d-85d6-dd2f52b2cb3a None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:25.015788 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:25.064257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd64c1-3b67-4c2d-85d6-dd2f52b2cb3a None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:25.117621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952228ae-fc9f-452b-82a6-0f66e5a9c522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:25.118264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952228ae-fc9f-452b-82a6-0f66e5a9c522 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:25.118604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952228ae-fc9f-452b-82a6-0f66e5a9c522 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:25.118876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952228ae-fc9f-452b-82a6-0f66e5a9c522 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:25.130752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952228ae-fc9f-452b-82a6-0f66e5a9c522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:25.132243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952228ae-fc9f-452b-82a6-0f66e5a9c522 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-952228ae-fc9f-452b-82a6-0f66e5a9c522 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:25.149424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:25.150441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:25.152608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:25.152608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:25.152608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:25.156221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 530/1068] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:25.157423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224ac18-a5b0-4980-b75b-c5f636d03c37 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:25.283175 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224ac18-a5b0-4980-b75b-c5f636d03c37 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:25.324899 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae887a8-f2b7-40b6-8666-8c3e6acac195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:25.325518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae887a8-f2b7-40b6-8666-8c3e6acac195 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:25.325793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae887a8-f2b7-40b6-8666-8c3e6acac195 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:25.326062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae887a8-f2b7-40b6-8666-8c3e6acac195 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:25.326708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae887a8-f2b7-40b6-8666-8c3e6acac195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:25.327008 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae887a8-f2b7-40b6-8666-8c3e6acac195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:25.328521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae887a8-f2b7-40b6-8666-8c3e6acac195 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:25.331150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:25.347656 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 539/1069] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:25.349198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952228ae-fc9f-452b-82a6-0f66e5a9c522 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:25.489265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952228ae-fc9f-452b-82a6-0f66e5a9c522 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:25.513683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:25.539464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7544b1-01f7-4a97-b3a9-54351c899f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:25.540388 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7544b1-01f7-4a97-b3a9-54351c899f27 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:25.540814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7544b1-01f7-4a97-b3a9-54351c899f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:25.541233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7544b1-01f7-4a97-b3a9-54351c899f27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:25.549802 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7544b1-01f7-4a97-b3a9-54351c899f27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:25.551468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7544b1-01f7-4a97-b3a9-54351c899f27 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-6b7544b1-01f7-4a97-b3a9-54351c899f27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:04:25.564023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:25.565122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:25.566244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:04:25.566244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:04:25.566244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:04:25.566244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:04:25.566244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:04:25.566244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:04:25.566244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:25.566244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:25.568244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 531/1070] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:04:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:25.573699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae887a8-f2b7-40b6-8666-8c3e6acac195 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:25.687354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae887a8-f2b7-40b6-8666-8c3e6acac195 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:25.734955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637476ab-aa8d-4aed-9358-601ca4db723c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:25.735568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637476ab-aa8d-4aed-9358-601ca4db723c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:25.735841 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637476ab-aa8d-4aed-9358-601ca4db723c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:25.736142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637476ab-aa8d-4aed-9358-601ca4db723c None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:25.736836 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637476ab-aa8d-4aed-9358-601ca4db723c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:25.737139 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637476ab-aa8d-4aed-9358-601ca4db723c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:25.738570 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637476ab-aa8d-4aed-9358-601ca4db723c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:25.756116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 540/1071] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:04:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:25.758212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7544b1-01f7-4a97-b3a9-54351c899f27 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:25.865753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e6245-3237-4da2-90c4-d162845634f0 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:25.866244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e6245-3237-4da2-90c4-d162845634f0 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:25.866244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e6245-3237-4da2-90c4-d162845634f0 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:25.867706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 541/1072] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:25.869359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7544b1-01f7-4a97-b3a9-54351c899f27 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:25.873232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d41359-7a3d-49a0-b817-96befafc489c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:25.873504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d41359-7a3d-49a0-b817-96befafc489c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:25.873771 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d41359-7a3d-49a0-b817-96befafc489c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:25.892625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e3d41359-7a3d-49a0-b817-96befafc489c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:25.893802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e3d41359-7a3d-49a0-b817-96befafc489c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:25.908984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678b2b8-fb3c-4e55-89bf-377037251ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:25.909925 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678b2b8-fb3c-4e55-89bf-377037251ae6 None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:25.910269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678b2b8-fb3c-4e55-89bf-377037251ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:25.910497 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678b2b8-fb3c-4e55-89bf-377037251ae6 None admin] PATH_INFO: `/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/049399a752ed4646a13f8e4a3a62d8e9/roles/71b1513afb19498a9ac3d5584d85750a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:25.913118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678b2b8-fb3c-4e55-89bf-377037251ae6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9188023bf51d4737b801e3fbd9af95b1, user_id=049399a752ed4646a13f8e4a3a62d8e9, role_id=71b1513afb19498a9ac3d5584d85750a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:25.915031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678b2b8-fb3c-4e55-89bf-377037251ae6 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:25.932996 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 532/1073] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:04:25 2022] PUT /identity/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/049399a752ed4646a13f8e4a3a62d8e9/roles/71b1513afb19498a9ac3d5584d85750a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:04:25.954777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:25.975256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 542/1074] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:26.254067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678b2b8-fb3c-4e55-89bf-377037251ae6 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:26.295380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:26.368086 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678b2b8-fb3c-4e55-89bf-377037251ae6 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:26.410001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d241-53f8-4bbd-92b2-cb5360adc3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:26.410640 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d241-53f8-4bbd-92b2-cb5360adc3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:26.410916 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d241-53f8-4bbd-92b2-cb5360adc3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:26.411184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d241-53f8-4bbd-92b2-cb5360adc3a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:26.423625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4d241-53f8-4bbd-92b2-cb5360adc3a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-3ae4d241-53f8-4bbd-92b2-cb5360adc3a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:04:26.428636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:26.429646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:26.436409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 533/1075] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:04:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:26.438739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d41359-7a3d-49a0-b817-96befafc489c None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:26.548472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d41359-7a3d-49a0-b817-96befafc489c None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:26.584329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293cd39-5085-443b-824a-18cf7224b71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:26.584955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293cd39-5085-443b-824a-18cf7224b71a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:26.585230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293cd39-5085-443b-824a-18cf7224b71a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:26.585499 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293cd39-5085-443b-824a-18cf7224b71a None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:26.586238 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293cd39-5085-443b-824a-18cf7224b71a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:26.586539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293cd39-5085-443b-824a-18cf7224b71a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:26.587957 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293cd39-5085-443b-824a-18cf7224b71a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:26.602212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 543/1076] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:04:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:26.604556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d241-53f8-4bbd-92b2-cb5360adc3a1 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:26.705979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:26.719159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d241-53f8-4bbd-92b2-cb5360adc3a1 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:26.755631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c43bd-848b-4f67-9f32-0875746ad964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:26.756295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c43bd-848b-4f67-9f32-0875746ad964 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:26.756603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c43bd-848b-4f67-9f32-0875746ad964 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:26.756906 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c43bd-848b-4f67-9f32-0875746ad964 None admin] PATH_INFO: `/v3/users/7cffb5e6721142cea4990b4be58bff7e` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:26.758040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c43bd-848b-4f67-9f32-0875746ad964 None admin] RBAC: Authorizing `identity:get_user(user_id=7cffb5e6721142cea4990b4be58bff7e)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:26.759466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c43bd-848b-4f67-9f32-0875746ad964 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:26.761810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 534/1077] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:04:26 2022] GET /identity/v3/users/7cffb5e6721142cea4990b4be58bff7e => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:26.767634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293cd39-5085-443b-824a-18cf7224b71a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:26.826261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:26.831866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edc045-b6e6-48c9-9887-d40b89c96759 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:26.832178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edc045-b6e6-48c9-9887-d40b89c96759 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:26.832430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edc045-b6e6-48c9-9887-d40b89c96759 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:26.834130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 535/1078] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:26.840660 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcab228-18df-41fb-9382-24b960d628d0 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:26.841113 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcab228-18df-41fb-9382-24b960d628d0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:26.841388 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcab228-18df-41fb-9382-24b960d628d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:26.865548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-5fcab228-18df-41fb-9382-24b960d628d0 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:26.866735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5fcab228-18df-41fb-9382-24b960d628d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:26.887947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293cd39-5085-443b-824a-18cf7224b71a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:26.926529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4936243-0fd4-4fa4-bc51-ab776edc0914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:26.927214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4936243-0fd4-4fa4-bc51-ab776edc0914 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:26.927544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4936243-0fd4-4fa4-bc51-ab776edc0914 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:26.927756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4936243-0fd4-4fa4-bc51-ab776edc0914 None admin] PATH_INFO: `/v3/projects/3b91df1085544a21ba9adf02b065c040` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:26.928990 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4936243-0fd4-4fa4-bc51-ab776edc0914 None admin] RBAC: Authorizing `identity:get_project(project_id=3b91df1085544a21ba9adf02b065c040)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:26.930725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4936243-0fd4-4fa4-bc51-ab776edc0914 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:26.937655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 544/1079] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:04:26 2022] GET /identity/v3/projects/3b91df1085544a21ba9adf02b065c040 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:26.939518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4936243-0fd4-4fa4-bc51-ab776edc0914 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:26.946271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 536/1080] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:27.049611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4936243-0fd4-4fa4-bc51-ab776edc0914 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:27.066545 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:27.087140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90c232-6ada-405c-a473-1c355dbdcbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:27.087724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90c232-6ada-405c-a473-1c355dbdcbfe None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:27.088028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90c232-6ada-405c-a473-1c355dbdcbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:27.089029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90c232-6ada-405c-a473-1c355dbdcbfe None admin] PATH_INFO: `/v3/projects/3b91df1085544a21ba9adf02b065c040/users/7cffb5e6721142cea4990b4be58bff7e/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:27.091085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90c232-6ada-405c-a473-1c355dbdcbfe None admin] RBAC: Authorizing `identity:create_grant(project_id=3b91df1085544a21ba9adf02b065c040, user_id=7cffb5e6721142cea4990b4be58bff7e, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:27.092594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90c232-6ada-405c-a473-1c355dbdcbfe None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:27.108248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 545/1081] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:04:26 2022] PUT /identity/v3/projects/3b91df1085544a21ba9adf02b065c040/users/7cffb5e6721142cea4990b4be58bff7e/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:04:27.263025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcab228-18df-41fb-9382-24b960d628d0 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:27.361941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:27.372402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcab228-18df-41fb-9382-24b960d628d0 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:27.409422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b80966-7e4c-425a-8058-680a77a64a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:27.410042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b80966-7e4c-425a-8058-680a77a64a09 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:27.410317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b80966-7e4c-425a-8058-680a77a64a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:27.410582 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b80966-7e4c-425a-8058-680a77a64a09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:27.489773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b80966-7e4c-425a-8058-680a77a64a09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:27.491322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b80966-7e4c-425a-8058-680a77a64a09 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:27.594524 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:27.606296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 537/1082] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:04:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:04:27.612421 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90c232-6ada-405c-a473-1c355dbdcbfe None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:27.729788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90c232-6ada-405c-a473-1c355dbdcbfe None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:27.767584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab11cf-83f2-4aa2-a347-6bea3ab81151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:27.768245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab11cf-83f2-4aa2-a347-6bea3ab81151 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:27.768526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab11cf-83f2-4aa2-a347-6bea3ab81151 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:27.768789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab11cf-83f2-4aa2-a347-6bea3ab81151 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:27.779801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab11cf-83f2-4aa2-a347-6bea3ab81151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:27.781301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab11cf-83f2-4aa2-a347-6bea3ab81151 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-a8ab11cf-83f2-4aa2-a347-6bea3ab81151 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:27.796177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:27.797274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:27.798326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:27.798326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:27.798326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:27.800351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 546/1083] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:27.803347 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b80966-7e4c-425a-8058-680a77a64a09 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:27.894068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa117b37-62a6-4928-b0c8-612fbde46d7d None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:27.894574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa117b37-62a6-4928-b0c8-612fbde46d7d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:27.894574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa117b37-62a6-4928-b0c8-612fbde46d7d None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:27.896032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 547/1084] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:27.900833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bf1b0-351d-49a3-b8c3-6cf8bd664704 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:27.901586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bf1b0-351d-49a3-b8c3-6cf8bd664704 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:27.901586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bf1b0-351d-49a3-b8c3-6cf8bd664704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:27.907044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b80966-7e4c-425a-8058-680a77a64a09 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:27.920665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-408bf1b0-351d-49a3-b8c3-6cf8bd664704 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:27.921860 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-408bf1b0-351d-49a3-b8c3-6cf8bd664704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:27.943846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261714b-df90-4292-8430-3f84dc9c3000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:27.944487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261714b-df90-4292-8430-3f84dc9c3000 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:27.944764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261714b-df90-4292-8430-3f84dc9c3000 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:27.945035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261714b-df90-4292-8430-3f84dc9c3000 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:27.945705 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261714b-df90-4292-8430-3f84dc9c3000 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:27.945983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261714b-df90-4292-8430-3f84dc9c3000 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:27.947352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261714b-df90-4292-8430-3f84dc9c3000 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:27.948840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:27.968028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 538/1085] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:28.006530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 548/1086] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:28.281986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261714b-df90-4292-8430-3f84dc9c3000 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:28.335377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:28.382570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261714b-df90-4292-8430-3f84dc9c3000 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:28.415920 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fd7c2-b54a-4f77-9e14-3418ac970bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:28.417355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fd7c2-b54a-4f77-9e14-3418ac970bdb None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:28.417630 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fd7c2-b54a-4f77-9e14-3418ac970bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:28.417900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fd7c2-b54a-4f77-9e14-3418ac970bdb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:28.428187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fd7c2-b54a-4f77-9e14-3418ac970bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-6f2fd7c2-b54a-4f77-9e14-3418ac970bdb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:04:28.433453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:28.434549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:28.435882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:28.435882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:28.435882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:28.435882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:28.435882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:04:28.435882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:04:28.435882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:04:28.435882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:04:28.435882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:04:28.435882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:04:28.435882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:04:28.435882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:28.439519 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 539/1087] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Aug 30 01:04:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:28.440628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bf1b0-351d-49a3-b8c3-6cf8bd664704 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:28.541040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bf1b0-351d-49a3-b8c3-6cf8bd664704 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:28.576236 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55670ba-20bd-4320-9f80-ed0bb5a5fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:28.576842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55670ba-20bd-4320-9f80-ed0bb5a5fa82 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:28.577113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55670ba-20bd-4320-9f80-ed0bb5a5fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:28.577375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55670ba-20bd-4320-9f80-ed0bb5a5fa82 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:28.578044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55670ba-20bd-4320-9f80-ed0bb5a5fa82 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:28.578335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55670ba-20bd-4320-9f80-ed0bb5a5fa82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:28.579697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55670ba-20bd-4320-9f80-ed0bb5a5fa82 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:28.587686 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 549/1088] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Aug 30 01:04:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:28.594151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fd7c2-b54a-4f77-9e14-3418ac970bdb None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:28.699779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fd7c2-b54a-4f77-9e14-3418ac970bdb None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:28.713015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:28.735119 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5d5af-69cc-4010-b7ee-77541afe867c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:28.735746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5d5af-69cc-4010-b7ee-77541afe867c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:28.736048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5d5af-69cc-4010-b7ee-77541afe867c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:28.736322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5d5af-69cc-4010-b7ee-77541afe867c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:28.749775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5d5af-69cc-4010-b7ee-77541afe867c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:28.751228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5d5af-69cc-4010-b7ee-77541afe867c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-b1f5d5af-69cc-4010-b7ee-77541afe867c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:28.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:28.766981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:28.768237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:28.768237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:28.768237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:28.768237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 540/1089] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:28.772912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55670ba-20bd-4320-9f80-ed0bb5a5fa82 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:28.797286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:28.877694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55670ba-20bd-4320-9f80-ed0bb5a5fa82 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:28.912725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127ae3b-d79e-4b2d-8eb2-10336a36d940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:28.914451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ae3b-d79e-4b2d-8eb2-10336a36d940 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:28.914451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ae3b-d79e-4b2d-8eb2-10336a36d940 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:28.914451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127ae3b-d79e-4b2d-8eb2-10336a36d940 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:28.914451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127ae3b-d79e-4b2d-8eb2-10336a36d940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:28.914853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127ae3b-d79e-4b2d-8eb2-10336a36d940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:28.915898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127ae3b-d79e-4b2d-8eb2-10336a36d940 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:28.935538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 550/1090] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:28.937013 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5d5af-69cc-4010-b7ee-77541afe867c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:29.039100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5d5af-69cc-4010-b7ee-77541afe867c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:29.073195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395a48c-f76e-4bec-9e98-6fd49efdf37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:29.073738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395a48c-f76e-4bec-9e98-6fd49efdf37d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:29.074012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395a48c-f76e-4bec-9e98-6fd49efdf37d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:29.074302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395a48c-f76e-4bec-9e98-6fd49efdf37d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:29.081993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395a48c-f76e-4bec-9e98-6fd49efdf37d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:29.083418 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395a48c-f76e-4bec-9e98-6fd49efdf37d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-6395a48c-f76e-4bec-9e98-6fd49efdf37d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:04:29.094874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:29.096111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:29.097047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:29.097047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:29.097047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:04:29.097047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:04:29.097047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:04:29.097047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:04:29.097047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:04:29.097047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:04:29.097047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:04:29.097047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:29.097047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 541/1091] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Aug 30 01:04:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:29.101974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127ae3b-d79e-4b2d-8eb2-10336a36d940 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:29.166472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d270279-bbbb-4b64-a441-acccb1dbe584 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:29.167039 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d270279-bbbb-4b64-a441-acccb1dbe584 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:29.167425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d270279-bbbb-4b64-a441-acccb1dbe584 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:29.168804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 542/1092] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:29.173799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2100e-4dee-43d1-80c0-db68d7df26bd None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:29.174076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2100e-4dee-43d1-80c0-db68d7df26bd None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:29.174348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2100e-4dee-43d1-80c0-db68d7df26bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:29.189210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:29.192752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c3d2100e-4dee-43d1-80c0-db68d7df26bd None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:29.193958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c3d2100e-4dee-43d1-80c0-db68d7df26bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:29.210912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127ae3b-d79e-4b2d-8eb2-10336a36d940 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:29.213666 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:29.246918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e236fa-cd34-4871-af4b-8a0f0968c16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:29.247524 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e236fa-cd34-4871-af4b-8a0f0968c16a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:29.247795 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e236fa-cd34-4871-af4b-8a0f0968c16a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:29.248100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e236fa-cd34-4871-af4b-8a0f0968c16a None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:29.248819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e236fa-cd34-4871-af4b-8a0f0968c16a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:29.249122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e236fa-cd34-4871-af4b-8a0f0968c16a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:29.250545 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e236fa-cd34-4871-af4b-8a0f0968c16a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:29.265466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 543/1093] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:29.273218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 551/1094] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Aug 30 01:04:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:29.274598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2100e-4dee-43d1-80c0-db68d7df26bd None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:29.381829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2100e-4dee-43d1-80c0-db68d7df26bd None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:29.420872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7499013-2a3c-4868-8f72-f7fcba9c58dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:29.421552 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7499013-2a3c-4868-8f72-f7fcba9c58dd None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:29.421763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7499013-2a3c-4868-8f72-f7fcba9c58dd None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:29.422029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7499013-2a3c-4868-8f72-f7fcba9c58dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:29.447917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:29.492778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7499013-2a3c-4868-8f72-f7fcba9c58dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:29.494315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7499013-2a3c-4868-8f72-f7fcba9c58dd None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:29.537665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e236fa-cd34-4871-af4b-8a0f0968c16a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:29.595608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 544/1095] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:04:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:04:29.601930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7499013-2a3c-4868-8f72-f7fcba9c58dd None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:29.642522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e236fa-cd34-4871-af4b-8a0f0968c16a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:29.660192 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:29.678058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6ddb8-af49-4df7-97d8-6c5c9e3d4232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:29.678683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6ddb8-af49-4df7-97d8-6c5c9e3d4232 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:29.678951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6ddb8-af49-4df7-97d8-6c5c9e3d4232 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:29.679211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6ddb8-af49-4df7-97d8-6c5c9e3d4232 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:29.687412 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:29.692438 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6ddb8-af49-4df7-97d8-6c5c9e3d4232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-fdd6ddb8-af49-4df7-97d8-6c5c9e3d4232 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:04:29.697768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:29.698830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:29.699682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 552/1096] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:04:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:29.705144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6ddb8-af49-4df7-97d8-6c5c9e3d4232 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:29.709557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7499013-2a3c-4868-8f72-f7fcba9c58dd None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:29.746213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223f796-8f53-46b6-a5df-43c17bd312b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:29.746851 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223f796-8f53-46b6-a5df-43c17bd312b5 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:29.747229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223f796-8f53-46b6-a5df-43c17bd312b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:29.747398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223f796-8f53-46b6-a5df-43c17bd312b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:29.758070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223f796-8f53-46b6-a5df-43c17bd312b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:29.759563 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223f796-8f53-46b6-a5df-43c17bd312b5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-b223f796-8f53-46b6-a5df-43c17bd312b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:04:29.775715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:29.776969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:04:29.777854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:04:29.777854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:04:29.777854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:29.777854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 545/1097] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:29.783941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223f796-8f53-46b6-a5df-43c17bd312b5 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:29.813549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6ddb8-af49-4df7-97d8-6c5c9e3d4232 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:29.849214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42b8df-8fea-4d56-9c12-6e107db6d967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:29.849827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42b8df-8fea-4d56-9c12-6e107db6d967 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:29.850107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42b8df-8fea-4d56-9c12-6e107db6d967 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:29.850380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42b8df-8fea-4d56-9c12-6e107db6d967 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:29.851047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42b8df-8fea-4d56-9c12-6e107db6d967 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:29.851350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42b8df-8fea-4d56-9c12-6e107db6d967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:29.852756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42b8df-8fea-4d56-9c12-6e107db6d967 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:29.861246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 553/1098] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:04:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:29.867246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42b8df-8fea-4d56-9c12-6e107db6d967 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:29.895076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223f796-8f53-46b6-a5df-43c17bd312b5 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:29.902509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:29.931684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8655d4-ce09-4cf6-ba90-423e433825b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:29.933064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8655d4-ce09-4cf6-ba90-423e433825b2 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:29.933477 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8655d4-ce09-4cf6-ba90-423e433825b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:29.933628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8655d4-ce09-4cf6-ba90-423e433825b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:29.934266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8655d4-ce09-4cf6-ba90-423e433825b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:29.934569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8655d4-ce09-4cf6-ba90-423e433825b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:29.935142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:29.935948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8655d4-ce09-4cf6-ba90-423e433825b2 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:29.948378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 546/1099] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:29.954026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8655d4-ce09-4cf6-ba90-423e433825b2 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:29.983292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42b8df-8fea-4d56-9c12-6e107db6d967 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:30.017443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44ad29-c9b0-4d7d-9f14-60c8f34e42f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:30.018066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44ad29-c9b0-4d7d-9f14-60c8f34e42f0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:30.018339 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44ad29-c9b0-4d7d-9f14-60c8f34e42f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:30.018602 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44ad29-c9b0-4d7d-9f14-60c8f34e42f0 None admin] PATH_INFO: `/v3/users/7cffb5e6721142cea4990b4be58bff7e` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:30.019729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44ad29-c9b0-4d7d-9f14-60c8f34e42f0 None admin] RBAC: Authorizing `identity:get_user(user_id=7cffb5e6721142cea4990b4be58bff7e)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:30.021203 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44ad29-c9b0-4d7d-9f14-60c8f34e42f0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:30.027809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 554/1100] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:04:29 2022] GET /identity/v3/users/7cffb5e6721142cea4990b4be58bff7e => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:30.028848 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44ad29-c9b0-4d7d-9f14-60c8f34e42f0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:30.057776 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8655d4-ce09-4cf6-ba90-423e433825b2 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:30.094041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089055c-239a-410c-8970-a0d9e5c323db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:30.094661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089055c-239a-410c-8970-a0d9e5c323db None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:30.094931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089055c-239a-410c-8970-a0d9e5c323db None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:30.095248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089055c-239a-410c-8970-a0d9e5c323db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:30.101950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089055c-239a-410c-8970-a0d9e5c323db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:30.103380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089055c-239a-410c-8970-a0d9e5c323db None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-7089055c-239a-410c-8970-a0d9e5c323db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:04:30.115060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:30.116115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:30.117025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:04:30.117025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:04:30.117025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:04:30.117025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:04:30.117025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:04:30.117025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:04:30.117025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:04:30.117025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:30.117025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 547/1101] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:04:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:04:30.122331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089055c-239a-410c-8970-a0d9e5c323db None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:30.138004 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44ad29-c9b0-4d7d-9f14-60c8f34e42f0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:30.150330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:30.166194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:30.175534 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d866ace-a787-440b-b174-2d269f10789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:30.176354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d866ace-a787-440b-b174-2d269f10789a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:30.176495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d866ace-a787-440b-b174-2d269f10789a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:30.176719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d866ace-a787-440b-b174-2d269f10789a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:30.233033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089055c-239a-410c-8970-a0d9e5c323db None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:30.250789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d866ace-a787-440b-b174-2d269f10789a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:30.252306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d866ace-a787-440b-b174-2d269f10789a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:30.271840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1411c2-70f7-4313-b46c-2bd7da6cabea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:30.272478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1411c2-70f7-4313-b46c-2bd7da6cabea None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:30.272754 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1411c2-70f7-4313-b46c-2bd7da6cabea None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:30.273031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1411c2-70f7-4313-b46c-2bd7da6cabea None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:30.274559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1411c2-70f7-4313-b46c-2bd7da6cabea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:30.274868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1411c2-70f7-4313-b46c-2bd7da6cabea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:30.277542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1411c2-70f7-4313-b46c-2bd7da6cabea None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:30.289453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 548/1102] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:04:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:30.296440 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1411c2-70f7-4313-b46c-2bd7da6cabea None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:30.358262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 555/1103] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:04:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:04:30.364109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d866ace-a787-440b-b174-2d269f10789a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:30.381966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:30.392129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:30.398350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1411c2-70f7-4313-b46c-2bd7da6cabea None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:30.437066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22661d45-70a9-4907-8730-8fbb3dd33ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:30.437717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22661d45-70a9-4907-8730-8fbb3dd33ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:30.438106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22661d45-70a9-4907-8730-8fbb3dd33ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:30.438380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22661d45-70a9-4907-8730-8fbb3dd33ea4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:30.439110 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22661d45-70a9-4907-8730-8fbb3dd33ea4 None admin] RBAC: Adding query filter params (role.id=71b1513afb19498a9ac3d5584d85750a, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=049399a752ed4646a13f8e4a3a62d8e9) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:30.439413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22661d45-70a9-4907-8730-8fbb3dd33ea4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:30.440872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22661d45-70a9-4907-8730-8fbb3dd33ea4 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:30.446981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 549/1104] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:04:30 2022] GET /identity/v3/role_assignments?user.id=049399a752ed4646a13f8e4a3a62d8e9&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=71b1513afb19498a9ac3d5584d85750a => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:30.471017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d866ace-a787-440b-b174-2d269f10789a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:30.510565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f97a2a-54b0-4111-b754-be45180c846f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:30.511027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f97a2a-54b0-4111-b754-be45180c846f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:30.511311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f97a2a-54b0-4111-b754-be45180c846f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:30.511585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f97a2a-54b0-4111-b754-be45180c846f None admin] PATH_INFO: `/v3/projects/3b91df1085544a21ba9adf02b065c040` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:30.512837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f97a2a-54b0-4111-b754-be45180c846f None admin] RBAC: Authorizing `identity:get_project(project_id=3b91df1085544a21ba9adf02b065c040)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:30.514307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f97a2a-54b0-4111-b754-be45180c846f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:30.516082 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 556/1105] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:04:30 2022] GET /identity/v3/projects/3b91df1085544a21ba9adf02b065c040 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:30.522430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22661d45-70a9-4907-8730-8fbb3dd33ea4 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:30.636140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22661d45-70a9-4907-8730-8fbb3dd33ea4 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:30.677940 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0362a-1cef-49a6-87af-b010d892c57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:30.678572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0362a-1cef-49a6-87af-b010d892c57c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:30.678855 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0362a-1cef-49a6-87af-b010d892c57c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:30.679129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0362a-1cef-49a6-87af-b010d892c57c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:30.680017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0362a-1cef-49a6-87af-b010d892c57c None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=3b91df1085544a21ba9adf02b065c040, user.id=7cffb5e6721142cea4990b4be58bff7e) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:30.680423 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0362a-1cef-49a6-87af-b010d892c57c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:30.682085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0362a-1cef-49a6-87af-b010d892c57c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:30.690204 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 550/1106] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:04:30 2022] GET /identity/v3/role_assignments?user.id=7cffb5e6721142cea4990b4be58bff7e&scope.project.id=3b91df1085544a21ba9adf02b065c040&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:32.700962 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510051a-c788-4006-ad3b-7f6535578454 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:32.701520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510051a-c788-4006-ad3b-7f6535578454 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:32.701611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510051a-c788-4006-ad3b-7f6535578454 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:32.702994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 557/1107] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:32.708575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4192e4a-a243-415d-ae25-7e70166a88d8 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:32.709869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4192e4a-a243-415d-ae25-7e70166a88d8 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:32.709869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4192e4a-a243-415d-ae25-7e70166a88d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:32.730219 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d4192e4a-a243-415d-ae25-7e70166a88d8 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:32.731401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d4192e4a-a243-415d-ae25-7e70166a88d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:32.741236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:32.810364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 551/1108] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:33.101335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510051a-c788-4006-ad3b-7f6535578454 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:33.200070 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:33.215441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510051a-c788-4006-ad3b-7f6535578454 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:33.251263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86d94e-d60c-4cf8-b924-4cba0fa2b328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:33.251891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86d94e-d60c-4cf8-b924-4cba0fa2b328 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:33.252196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86d94e-d60c-4cf8-b924-4cba0fa2b328 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:33.252467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86d94e-d60c-4cf8-b924-4cba0fa2b328 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:33.253567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86d94e-d60c-4cf8-b924-4cba0fa2b328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:33.255018 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86d94e-d60c-4cf8-b924-4cba0fa2b328 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:33.256631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 558/1109] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:33.264079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4192e4a-a243-415d-ae25-7e70166a88d8 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:33.368042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4192e4a-a243-415d-ae25-7e70166a88d8 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:33.402491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a857d-be1f-46f0-bb3f-d6dcf1cf31ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:33.403098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a857d-be1f-46f0-bb3f-d6dcf1cf31ee None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:33.403372 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a857d-be1f-46f0-bb3f-d6dcf1cf31ee None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:33.403639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a857d-be1f-46f0-bb3f-d6dcf1cf31ee None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:33.404332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a857d-be1f-46f0-bb3f-d6dcf1cf31ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:33.405746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a857d-be1f-46f0-bb3f-d6dcf1cf31ee None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:33.422491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:33.433201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 552/1110] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:04:33 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:04:35.419408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5236626-a5cd-4d6e-bf1c-89c41a1cf416 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:35.419939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5236626-a5cd-4d6e-bf1c-89c41a1cf416 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:35.419939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5236626-a5cd-4d6e-bf1c-89c41a1cf416 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:35.421329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 559/1111] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:35.426537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ca4f9-688a-40e6-888f-a9736a58cbe7 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:35.426984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ca4f9-688a-40e6-888f-a9736a58cbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:35.427074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ca4f9-688a-40e6-888f-a9736a58cbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:35.445264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-647ca4f9-688a-40e6-888f-a9736a58cbe7 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:35.446449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-647ca4f9-688a-40e6-888f-a9736a58cbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:35.514120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 553/1112] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:35.784846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5236626-a5cd-4d6e-bf1c-89c41a1cf416 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:35.889349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5236626-a5cd-4d6e-bf1c-89c41a1cf416 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:35.926058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11589d32-a15a-43b2-875a-5a05fbd2b2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:35.926913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11589d32-a15a-43b2-875a-5a05fbd2b2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:35.927215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11589d32-a15a-43b2-875a-5a05fbd2b2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:35.927799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11589d32-a15a-43b2-875a-5a05fbd2b2b7 None admin] PATH_INFO: `/v3/roles/7562948bd63d483cad68427819d434a9` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:35.930268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11589d32-a15a-43b2-875a-5a05fbd2b2b7 None admin] RBAC: Authorizing `identity:get_role(role_id=7562948bd63d483cad68427819d434a9)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:35.932712 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11589d32-a15a-43b2-875a-5a05fbd2b2b7 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:35.934651 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 560/1113] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:04:35 2022] GET /identity/v3/roles/7562948bd63d483cad68427819d434a9 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:35.940880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ca4f9-688a-40e6-888f-a9736a58cbe7 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:36.040111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ca4f9-688a-40e6-888f-a9736a58cbe7 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:36.069841 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:36.076614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a03a0c-f4ce-4b6d-aee4-bfadc6556efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:36.077247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a03a0c-f4ce-4b6d-aee4-bfadc6556efa None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:36.077525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a03a0c-f4ce-4b6d-aee4-bfadc6556efa None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:36.077796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a03a0c-f4ce-4b6d-aee4-bfadc6556efa None admin] PATH_INFO: `/v3/users/2d1de288b4414ef4b2c2fcccb34e4071` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:36.091488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a03a0c-f4ce-4b6d-aee4-bfadc6556efa None admin] RBAC: Authorizing `identity:get_user(user_id=2d1de288b4414ef4b2c2fcccb34e4071)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:36.092963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a03a0c-f4ce-4b6d-aee4-bfadc6556efa None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:36.094605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 554/1114] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:04:35 2022] GET /identity/v3/users/2d1de288b4414ef4b2c2fcccb34e4071 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:36.100521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11589d32-a15a-43b2-875a-5a05fbd2b2b7 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:36.132663 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:36.205421 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11589d32-a15a-43b2-875a-5a05fbd2b2b7 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:36.241272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bdd55-6e96-4df7-a565-b6c3b51687fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:36.241888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bdd55-6e96-4df7-a565-b6c3b51687fd None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:36.242162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bdd55-6e96-4df7-a565-b6c3b51687fd None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:36.242431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bdd55-6e96-4df7-a565-b6c3b51687fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:36.313943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bdd55-6e96-4df7-a565-b6c3b51687fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:36.315435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bdd55-6e96-4df7-a565-b6c3b51687fd None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:36.351631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:36.420527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 561/1115] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:04:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:04:36.426731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a03a0c-f4ce-4b6d-aee4-bfadc6556efa None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:36.551873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a03a0c-f4ce-4b6d-aee4-bfadc6556efa None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:36.596259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13952dae-9c6f-4597-a95d-984c50946d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:36.596840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13952dae-9c6f-4597-a95d-984c50946d0f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:36.597119 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13952dae-9c6f-4597-a95d-984c50946d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:36.597392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13952dae-9c6f-4597-a95d-984c50946d0f None admin] PATH_INFO: `/v3/projects/3b91df1085544a21ba9adf02b065c040` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:36.598536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13952dae-9c6f-4597-a95d-984c50946d0f None admin] RBAC: Authorizing `identity:get_project(project_id=3b91df1085544a21ba9adf02b065c040)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:36.599973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13952dae-9c6f-4597-a95d-984c50946d0f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:36.601647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 555/1116] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:04:36 2022] GET /identity/v3/projects/3b91df1085544a21ba9adf02b065c040 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:36.608848 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bdd55-6e96-4df7-a565-b6c3b51687fd None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:36.714688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bdd55-6e96-4df7-a565-b6c3b51687fd None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:36.744700 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:36.750061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b7aa3-2865-41da-86bf-ae7266a33c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:36.750664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b7aa3-2865-41da-86bf-ae7266a33c10 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:36.750937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b7aa3-2865-41da-86bf-ae7266a33c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:36.751205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b7aa3-2865-41da-86bf-ae7266a33c10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:36.751926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b7aa3-2865-41da-86bf-ae7266a33c10 None admin] RBAC: Adding query filter params (role.id=7562948bd63d483cad68427819d434a9, scope.project.id=3b91df1085544a21ba9adf02b065c040, user.id=2d1de288b4414ef4b2c2fcccb34e4071) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:36.752366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b7aa3-2865-41da-86bf-ae7266a33c10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:36.753785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b7aa3-2865-41da-86bf-ae7266a33c10 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:36.760332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 562/1117] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:04:36 2022] GET /identity/v3/role_assignments?user.id=2d1de288b4414ef4b2c2fcccb34e4071&scope.project.id=3b91df1085544a21ba9adf02b065c040&role.id=7562948bd63d483cad68427819d434a9 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:38.702099 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0382-1d60-4e08-ba6b-915cec9d535e None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:38.702663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0382-1d60-4e08-ba6b-915cec9d535e None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:38.702663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0382-1d60-4e08-ba6b-915cec9d535e None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:38.704067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 556/1118] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:38.709766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cef48-7d2a-4dc0-943a-7912517ea3c4 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:38.710221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cef48-7d2a-4dc0-943a-7912517ea3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:38.710337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cef48-7d2a-4dc0-943a-7912517ea3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:38.728601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-1b5cef48-7d2a-4dc0-943a-7912517ea3c4 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:38.729788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1b5cef48-7d2a-4dc0-943a-7912517ea3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:38.797173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 563/1119] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:39.074867 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0382-1d60-4e08-ba6b-915cec9d535e None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:39.137758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:39.184111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0382-1d60-4e08-ba6b-915cec9d535e None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:39.218393 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cffc131-9c9c-4d0e-825f-b11d0d12108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:39.219024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffc131-9c9c-4d0e-825f-b11d0d12108a None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:39.219300 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffc131-9c9c-4d0e-825f-b11d0d12108a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:39.219567 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffc131-9c9c-4d0e-825f-b11d0d12108a None admin] PATH_INFO: `/v3/roles/7562948bd63d483cad68427819d434a9` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:39.221011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cffc131-9c9c-4d0e-825f-b11d0d12108a None admin] RBAC: Authorizing `identity:get_role(role_id=7562948bd63d483cad68427819d434a9)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:39.222388 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cffc131-9c9c-4d0e-825f-b11d0d12108a None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:39.223763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 557/1120] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:04:39 2022] GET /identity/v3/roles/7562948bd63d483cad68427819d434a9 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:39.230657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cef48-7d2a-4dc0-943a-7912517ea3c4 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:39.340444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cef48-7d2a-4dc0-943a-7912517ea3c4 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:39.365996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:39.375265 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4558e6a-842e-48b6-8e37-649f5466d622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:39.375891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4558e6a-842e-48b6-8e37-649f5466d622 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:39.376192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4558e6a-842e-48b6-8e37-649f5466d622 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:39.376460 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4558e6a-842e-48b6-8e37-649f5466d622 None admin] PATH_INFO: `/v3/users/2d1de288b4414ef4b2c2fcccb34e4071` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:39.377571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4558e6a-842e-48b6-8e37-649f5466d622 None admin] RBAC: Authorizing `identity:get_user(user_id=2d1de288b4414ef4b2c2fcccb34e4071)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:39.379131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4558e6a-842e-48b6-8e37-649f5466d622 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:39.380789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 564/1121] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:04:39 2022] GET /identity/v3/users/2d1de288b4414ef4b2c2fcccb34e4071 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:39.386650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cffc131-9c9c-4d0e-825f-b11d0d12108a None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:39.498569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cffc131-9c9c-4d0e-825f-b11d0d12108a None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:39.523243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:39.535497 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766f2a4-02b2-47d9-af8a-d50f95fa7538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:39.536157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766f2a4-02b2-47d9-af8a-d50f95fa7538 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:39.536433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766f2a4-02b2-47d9-af8a-d50f95fa7538 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:39.536700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766f2a4-02b2-47d9-af8a-d50f95fa7538 None admin] PATH_INFO: `/v3/projects/3b91df1085544a21ba9adf02b065c040` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:39.537810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766f2a4-02b2-47d9-af8a-d50f95fa7538 None admin] RBAC: Authorizing `identity:get_project(project_id=3b91df1085544a21ba9adf02b065c040)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:39.539230 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766f2a4-02b2-47d9-af8a-d50f95fa7538 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:39.541964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 558/1122] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:04:39 2022] GET /identity/v3/projects/3b91df1085544a21ba9adf02b065c040 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:39.548026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4558e6a-842e-48b6-8e37-649f5466d622 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:39.652358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4558e6a-842e-48b6-8e37-649f5466d622 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:39.688995 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279ecea-6bed-49b9-9bc3-2c1cbdd749bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:39.689672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279ecea-6bed-49b9-9bc3-2c1cbdd749bb None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:39.689948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279ecea-6bed-49b9-9bc3-2c1cbdd749bb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:39.690219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279ecea-6bed-49b9-9bc3-2c1cbdd749bb None admin] PATH_INFO: `/v3/projects/3b91df1085544a21ba9adf02b065c040/users/2d1de288b4414ef4b2c2fcccb34e4071/roles/7562948bd63d483cad68427819d434a9` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:39.692429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279ecea-6bed-49b9-9bc3-2c1cbdd749bb None admin] RBAC: Authorizing `identity:create_grant(project_id=3b91df1085544a21ba9adf02b065c040, user_id=2d1de288b4414ef4b2c2fcccb34e4071, role_id=7562948bd63d483cad68427819d434a9)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:39.693881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279ecea-6bed-49b9-9bc3-2c1cbdd749bb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:39.708896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 565/1123] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:04:39 2022] PUT /identity/v3/projects/3b91df1085544a21ba9adf02b065c040/users/2d1de288b4414ef4b2c2fcccb34e4071/roles/7562948bd63d483cad68427819d434a9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:04:41.904925 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f0f2c-ed5c-4897-9607-37a383e78000 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:41.905496 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f0f2c-ed5c-4897-9607-37a383e78000 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:41.905634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f0f2c-ed5c-4897-9607-37a383e78000 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:41.907167 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 559/1124] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:41.913241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dcbe1-779b-4315-9188-b5e1ce84a576 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:41.913769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dcbe1-779b-4315-9188-b5e1ce84a576 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:41.913908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dcbe1-779b-4315-9188-b5e1ce84a576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:41.934027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-580dcbe1-779b-4315-9188-b5e1ce84a576 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:41.935319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-580dcbe1-779b-4315-9188-b5e1ce84a576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:41.994687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:42.018842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 566/1125] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:42.307042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f0f2c-ed5c-4897-9607-37a383e78000 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:42.412835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f0f2c-ed5c-4897-9607-37a383e78000 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:42.450433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60cfcb-e171-4a8c-9891-cfd5b5bd7dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:42.451075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60cfcb-e171-4a8c-9891-cfd5b5bd7dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:42.451353 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60cfcb-e171-4a8c-9891-cfd5b5bd7dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:42.451656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60cfcb-e171-4a8c-9891-cfd5b5bd7dc7 None admin] PATH_INFO: `/v3/roles/7562948bd63d483cad68427819d434a9` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:42.453253 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60cfcb-e171-4a8c-9891-cfd5b5bd7dc7 None admin] RBAC: Authorizing `identity:get_role(role_id=7562948bd63d483cad68427819d434a9)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:42.454636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60cfcb-e171-4a8c-9891-cfd5b5bd7dc7 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:42.456117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 560/1126] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:04:42 2022] GET /identity/v3/roles/7562948bd63d483cad68427819d434a9 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:42.462857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dcbe1-779b-4315-9188-b5e1ce84a576 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:42.568563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dcbe1-779b-4315-9188-b5e1ce84a576 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:42.606609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70c127-875f-4717-b513-749c2e17dadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:42.607322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70c127-875f-4717-b513-749c2e17dadb None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:42.607646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70c127-875f-4717-b513-749c2e17dadb None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:42.608126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70c127-875f-4717-b513-749c2e17dadb None admin] PATH_INFO: `/v3/users/2d1de288b4414ef4b2c2fcccb34e4071` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:42.609428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70c127-875f-4717-b513-749c2e17dadb None admin] RBAC: Authorizing `identity:get_user(user_id=2d1de288b4414ef4b2c2fcccb34e4071)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:42.611050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70c127-875f-4717-b513-749c2e17dadb None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:42.612993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 567/1127] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:04:42 2022] GET /identity/v3/users/2d1de288b4414ef4b2c2fcccb34e4071 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:42.619799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60cfcb-e171-4a8c-9891-cfd5b5bd7dc7 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:42.693115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:42.736597 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60cfcb-e171-4a8c-9891-cfd5b5bd7dc7 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:42.774324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd449070-04ed-40cc-b6ad-34575f46f10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:42.775007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd449070-04ed-40cc-b6ad-34575f46f10d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:42.775306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd449070-04ed-40cc-b6ad-34575f46f10d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:42.775619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd449070-04ed-40cc-b6ad-34575f46f10d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:42.857229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd449070-04ed-40cc-b6ad-34575f46f10d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:42.858716 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd449070-04ed-40cc-b6ad-34575f46f10d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:42.921689 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:42.966880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 561/1128] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:04:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:04:42.973211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70c127-875f-4717-b513-749c2e17dadb None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:43.030792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:43.082720 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70c127-875f-4717-b513-749c2e17dadb None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:43.121751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9c5c6-a783-427c-9265-4419944f2a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:43.122308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9c5c6-a783-427c-9265-4419944f2a9c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:43.122603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9c5c6-a783-427c-9265-4419944f2a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:43.122877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9c5c6-a783-427c-9265-4419944f2a9c None admin] PATH_INFO: `/v3/projects/3b91df1085544a21ba9adf02b065c040` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:43.124108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9c5c6-a783-427c-9265-4419944f2a9c None admin] RBAC: Authorizing `identity:get_project(project_id=3b91df1085544a21ba9adf02b065c040)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:43.125624 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9c5c6-a783-427c-9265-4419944f2a9c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:43.127386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 568/1129] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:04:42 2022] GET /identity/v3/projects/3b91df1085544a21ba9adf02b065c040 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:43.137299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd449070-04ed-40cc-b6ad-34575f46f10d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:43.264571 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd449070-04ed-40cc-b6ad-34575f46f10d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:43.325545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5404c5-f624-4447-909d-89b847928805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:43.326532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5404c5-f624-4447-909d-89b847928805 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:43.327012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5404c5-f624-4447-909d-89b847928805 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:43.327491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5404c5-f624-4447-909d-89b847928805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:43.328833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5404c5-f624-4447-909d-89b847928805 None admin] RBAC: Adding query filter params (role.id=7562948bd63d483cad68427819d434a9, scope.project.id=3b91df1085544a21ba9adf02b065c040, user.id=2d1de288b4414ef4b2c2fcccb34e4071) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:43.329391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5404c5-f624-4447-909d-89b847928805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:43.332134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5404c5-f624-4447-909d-89b847928805 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:43.343289 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 562/1130] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:04:43 2022] GET /identity/v3/role_assignments?user.id=2d1de288b4414ef4b2c2fcccb34e4071&scope.project.id=3b91df1085544a21ba9adf02b065c040&role.id=7562948bd63d483cad68427819d434a9 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:45.355286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c2650-6ef3-41c7-aace-1f60eb319493 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:45.355892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c2650-6ef3-41c7-aace-1f60eb319493 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:45.355892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c2650-6ef3-41c7-aace-1f60eb319493 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:45.357313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 569/1131] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:45.363413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4035cda-6067-4fc2-be54-3d2a008d52d3 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:45.363946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4035cda-6067-4fc2-be54-3d2a008d52d3 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:45.363946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4035cda-6067-4fc2-be54-3d2a008d52d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:45.383137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-e4035cda-6067-4fc2-be54-3d2a008d52d3 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:45.384779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e4035cda-6067-4fc2-be54-3d2a008d52d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:45.404105 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:45.459757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 563/1132] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:45.742626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c2650-6ef3-41c7-aace-1f60eb319493 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:45.855936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c2650-6ef3-41c7-aace-1f60eb319493 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:45.878715 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:45.894164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353be3eb-12fc-4e5d-b4cb-3b19d8bc5a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:45.894804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353be3eb-12fc-4e5d-b4cb-3b19d8bc5a0d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:45.895078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353be3eb-12fc-4e5d-b4cb-3b19d8bc5a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:45.895344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353be3eb-12fc-4e5d-b4cb-3b19d8bc5a0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:45.896540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353be3eb-12fc-4e5d-b4cb-3b19d8bc5a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:45.898024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353be3eb-12fc-4e5d-b4cb-3b19d8bc5a0d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:45.900236 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 570/1133] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:45.908961 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4035cda-6067-4fc2-be54-3d2a008d52d3 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:46.025734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4035cda-6067-4fc2-be54-3d2a008d52d3 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:46.071249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8949d1-a98b-4f77-84a8-1f64ab2e880b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:46.072327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8949d1-a98b-4f77-84a8-1f64ab2e880b None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:46.072808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8949d1-a98b-4f77-84a8-1f64ab2e880b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:46.073255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8949d1-a98b-4f77-84a8-1f64ab2e880b None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:46.074397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8949d1-a98b-4f77-84a8-1f64ab2e880b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:46.077064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8949d1-a98b-4f77-84a8-1f64ab2e880b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:46.117359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 564/1134] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:04:45 2022] POST /identity/v3/projects => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:04:48.143831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2ca07-c268-4f1f-9900-03a9806e6504 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:48.144434 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2ca07-c268-4f1f-9900-03a9806e6504 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:48.144434 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2ca07-c268-4f1f-9900-03a9806e6504 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:48.145804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 571/1135] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:48.151375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38beddad-65ef-4a7c-9500-bb9d5225bbac None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:48.151926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38beddad-65ef-4a7c-9500-bb9d5225bbac None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:48.151926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38beddad-65ef-4a7c-9500-bb9d5225bbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:48.165268 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:48.171950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-38beddad-65ef-4a7c-9500-bb9d5225bbac None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:48.173381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-38beddad-65ef-4a7c-9500-bb9d5225bbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:48.257080 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 565/1136] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:48.538345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2ca07-c268-4f1f-9900-03a9806e6504 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:48.660584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2ca07-c268-4f1f-9900-03a9806e6504 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:48.707302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f2beb-9ba3-4f11-9834-33c1fe7a973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:48.707979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f2beb-9ba3-4f11-9834-33c1fe7a973f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:48.708281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f2beb-9ba3-4f11-9834-33c1fe7a973f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:48.708551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f2beb-9ba3-4f11-9834-33c1fe7a973f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:48.709903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f2beb-9ba3-4f11-9834-33c1fe7a973f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:48.712334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f2beb-9ba3-4f11-9834-33c1fe7a973f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:48.714083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 572/1137] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:04:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:48.721849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38beddad-65ef-4a7c-9500-bb9d5225bbac None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:48.839714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38beddad-65ef-4a7c-9500-bb9d5225bbac None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:48.876396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6371f7-ab4a-486e-b13c-cd9b47fffdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:48.877044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6371f7-ab4a-486e-b13c-cd9b47fffdfa None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:48.877328 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6371f7-ab4a-486e-b13c-cd9b47fffdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:48.877605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6371f7-ab4a-486e-b13c-cd9b47fffdfa None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:48.878280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6371f7-ab4a-486e-b13c-cd9b47fffdfa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:48.879731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6371f7-ab4a-486e-b13c-cd9b47fffdfa None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:48.890197 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:48.905423 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 566/1138] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:04:48 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:04:51.049794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974f01b-d71f-4597-b165-6bffb3027e34 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:51.050340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974f01b-d71f-4597-b165-6bffb3027e34 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:51.050340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974f01b-d71f-4597-b165-6bffb3027e34 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:51.051697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 573/1139] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:51.057862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555ae79-9e22-4601-9478-916c7896f3dc None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:51.058371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555ae79-9e22-4601-9478-916c7896f3dc None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:51.058480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555ae79-9e22-4601-9478-916c7896f3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:51.076832 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-0555ae79-9e22-4601-9478-916c7896f3dc None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:51.078037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0555ae79-9e22-4601-9478-916c7896f3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:51.151278 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 567/1140] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:04:51.437764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974f01b-d71f-4597-b165-6bffb3027e34 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:51.500867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:51.548416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974f01b-d71f-4597-b165-6bffb3027e34 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:51.584123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f9e9f-9f38-439e-9899-97a3128adc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:51.584745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f9e9f-9f38-439e-9899-97a3128adc94 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:51.585023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f9e9f-9f38-439e-9899-97a3128adc94 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:51.585291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f9e9f-9f38-439e-9899-97a3128adc94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:51.596055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f9e9f-9f38-439e-9899-97a3128adc94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-1d7f9e9f-9f38-439e-9899-97a3128adc94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:04:51.600809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:51.602111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:51.603198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:51.603198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:51.603198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:51.603198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:51.603198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:04:51.603198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:04:51.603198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:04:51.603198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:04:51.603198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:04:51.603198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:04:51.603198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:04:51.603198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:51.607745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 574/1141] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:04:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:51.608872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555ae79-9e22-4601-9478-916c7896f3dc None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:51.709966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555ae79-9e22-4601-9478-916c7896f3dc None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:51.733761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:51.747082 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c640d1-c78c-4d77-8df1-866b06794b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:51.747694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c640d1-c78c-4d77-8df1-866b06794b68 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:51.747971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c640d1-c78c-4d77-8df1-866b06794b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:51.748274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c640d1-c78c-4d77-8df1-866b06794b68 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:51.748933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c640d1-c78c-4d77-8df1-866b06794b68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:51.749234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c640d1-c78c-4d77-8df1-866b06794b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:51.750600 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c640d1-c78c-4d77-8df1-866b06794b68 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:51.759409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 568/1142] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:04:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:51.767547 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f9e9f-9f38-439e-9899-97a3128adc94 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:51.867842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f9e9f-9f38-439e-9899-97a3128adc94 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:51.884049 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:51.901897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d8259-e1b8-4df1-98ce-090b2f4946ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:51.902521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d8259-e1b8-4df1-98ce-090b2f4946ab None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:51.902793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d8259-e1b8-4df1-98ce-090b2f4946ab None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:51.903059 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d8259-e1b8-4df1-98ce-090b2f4946ab None admin] PATH_INFO: `/v3/users/357926d5d84f4c38a57a56b240a5d4ca` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:51.916972 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d8259-e1b8-4df1-98ce-090b2f4946ab None admin] RBAC: Authorizing `identity:get_user(user_id=357926d5d84f4c38a57a56b240a5d4ca)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:51.918398 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d8259-e1b8-4df1-98ce-090b2f4946ab None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:51.920010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 575/1143] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:04:51 2022] GET /identity/v3/users/357926d5d84f4c38a57a56b240a5d4ca => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:51.926129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c640d1-c78c-4d77-8df1-866b06794b68 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:52.025278 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c640d1-c78c-4d77-8df1-866b06794b68 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:52.058232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b105f22-98e9-4d83-9e0e-cf9a05f6971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:52.058872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b105f22-98e9-4d83-9e0e-cf9a05f6971d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:52.059089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b105f22-98e9-4d83-9e0e-cf9a05f6971d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:52.059355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b105f22-98e9-4d83-9e0e-cf9a05f6971d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:52.111039 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:52.127192 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b105f22-98e9-4d83-9e0e-cf9a05f6971d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:52.128664 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b105f22-98e9-4d83-9e0e-cf9a05f6971d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:52.244610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 569/1144] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:04:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:04:52.250887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d8259-e1b8-4df1-98ce-090b2f4946ab None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:52.350375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d8259-e1b8-4df1-98ce-090b2f4946ab None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:52.384243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a463d84-3575-490a-bab6-db8fc4bb9073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:52.384862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a463d84-3575-490a-bab6-db8fc4bb9073 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:52.385136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a463d84-3575-490a-bab6-db8fc4bb9073 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:52.385401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a463d84-3575-490a-bab6-db8fc4bb9073 None admin] PATH_INFO: `/v3/projects/c1a70fc4d1514b13af12318ccae4cc9b` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:52.386493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a463d84-3575-490a-bab6-db8fc4bb9073 None admin] RBAC: Authorizing `identity:get_project(project_id=c1a70fc4d1514b13af12318ccae4cc9b)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:52.387914 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a463d84-3575-490a-bab6-db8fc4bb9073 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:52.389562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 576/1145] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:04:52 2022] GET /identity/v3/projects/c1a70fc4d1514b13af12318ccae4cc9b => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:52.395790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b105f22-98e9-4d83-9e0e-cf9a05f6971d None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:52.481238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:52.497076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b105f22-98e9-4d83-9e0e-cf9a05f6971d None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:52.532674 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f79bee-fda1-4645-9821-5dc6d96145f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:52.533310 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f79bee-fda1-4645-9821-5dc6d96145f5 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:52.533584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f79bee-fda1-4645-9821-5dc6d96145f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:52.533849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f79bee-fda1-4645-9821-5dc6d96145f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:52.534573 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f79bee-fda1-4645-9821-5dc6d96145f5 None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=c1a70fc4d1514b13af12318ccae4cc9b, user.id=357926d5d84f4c38a57a56b240a5d4ca) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:52.534874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f79bee-fda1-4645-9821-5dc6d96145f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:52.536327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f79bee-fda1-4645-9821-5dc6d96145f5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:52.542170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 570/1146] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:04:52 2022] GET /identity/v3/role_assignments?user.id=357926d5d84f4c38a57a56b240a5d4ca&scope.project.id=c1a70fc4d1514b13af12318ccae4cc9b&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:54.418653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e3a7f-9cdf-41b4-8fbc-2611f8944f15 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:54.419192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e3a7f-9cdf-41b4-8fbc-2611f8944f15 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:54.419192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e3a7f-9cdf-41b4-8fbc-2611f8944f15 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:54.420570 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 577/1147] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:54.425835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42682fb-30cc-4cf6-ad36-00a6a0c3aead None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:54.426263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42682fb-30cc-4cf6-ad36-00a6a0c3aead None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:54.426385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42682fb-30cc-4cf6-ad36-00a6a0c3aead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:54.445875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-e42682fb-30cc-4cf6-ad36-00a6a0c3aead None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:54.447070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e42682fb-30cc-4cf6-ad36-00a6a0c3aead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:54.516196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 571/1148] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:54.789951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e3a7f-9cdf-41b4-8fbc-2611f8944f15 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:54.840368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:54.899433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e3a7f-9cdf-41b4-8fbc-2611f8944f15 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:54.935250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07892719-8e39-4a4a-8bec-5f5283826a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:54.935911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07892719-8e39-4a4a-8bec-5f5283826a7c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:54.936251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07892719-8e39-4a4a-8bec-5f5283826a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:54.936526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07892719-8e39-4a4a-8bec-5f5283826a7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:54.947451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07892719-8e39-4a4a-8bec-5f5283826a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-07892719-8e39-4a4a-8bec-5f5283826a7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:04:54.952525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:54.954043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:04:54.955678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 578/1149] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:04:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:54.960183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42682fb-30cc-4cf6-ad36-00a6a0c3aead None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:55.041856 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:55.064193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42682fb-30cc-4cf6-ad36-00a6a0c3aead None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:55.099420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1097fe-c114-40e3-9478-8414fb364b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:55.100052 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1097fe-c114-40e3-9478-8414fb364b1f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:55.100336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1097fe-c114-40e3-9478-8414fb364b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:55.100612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1097fe-c114-40e3-9478-8414fb364b1f None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:55.101418 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1097fe-c114-40e3-9478-8414fb364b1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:55.101771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1097fe-c114-40e3-9478-8414fb364b1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:55.103257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1097fe-c114-40e3-9478-8414fb364b1f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:55.111658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 572/1150] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:04:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:55.118228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07892719-8e39-4a4a-8bec-5f5283826a7c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:55.222098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07892719-8e39-4a4a-8bec-5f5283826a7c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:55.223656 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:55.258943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ccb96-25dc-4996-bbf0-296379bfe884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:55.259568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ccb96-25dc-4996-bbf0-296379bfe884 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:55.259847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ccb96-25dc-4996-bbf0-296379bfe884 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:55.260154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ccb96-25dc-4996-bbf0-296379bfe884 None admin] PATH_INFO: `/v3/users/357926d5d84f4c38a57a56b240a5d4ca` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:55.261274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ccb96-25dc-4996-bbf0-296379bfe884 None admin] RBAC: Authorizing `identity:get_user(user_id=357926d5d84f4c38a57a56b240a5d4ca)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:55.262703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ccb96-25dc-4996-bbf0-296379bfe884 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:55.264325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 579/1151] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:04:55 2022] GET /identity/v3/users/357926d5d84f4c38a57a56b240a5d4ca => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:55.270517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1097fe-c114-40e3-9478-8414fb364b1f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:55.373677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1097fe-c114-40e3-9478-8414fb364b1f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:55.407264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3fba0-3599-497c-9f5d-1d1ede02c57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:55.407885 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3fba0-3599-497c-9f5d-1d1ede02c57d None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:55.408192 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3fba0-3599-497c-9f5d-1d1ede02c57d None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:55.408464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3fba0-3599-497c-9f5d-1d1ede02c57d None admin] PATH_INFO: `/v3/projects/c1a70fc4d1514b13af12318ccae4cc9b` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:55.409568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3fba0-3599-497c-9f5d-1d1ede02c57d None admin] RBAC: Authorizing `identity:get_project(project_id=c1a70fc4d1514b13af12318ccae4cc9b)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:55.410992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3fba0-3599-497c-9f5d-1d1ede02c57d None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:55.412636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 573/1152] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:04:55 2022] GET /identity/v3/projects/c1a70fc4d1514b13af12318ccae4cc9b => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:04:55.419291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ccb96-25dc-4996-bbf0-296379bfe884 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:55.517469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ccb96-25dc-4996-bbf0-296379bfe884 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:55.555380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abc4cf-4f77-46b7-bf80-f24a9842b398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:55.556046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abc4cf-4f77-46b7-bf80-f24a9842b398 None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:55.556321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abc4cf-4f77-46b7-bf80-f24a9842b398 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:55.556589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abc4cf-4f77-46b7-bf80-f24a9842b398 None admin] PATH_INFO: `/v3/projects/c1a70fc4d1514b13af12318ccae4cc9b/users/357926d5d84f4c38a57a56b240a5d4ca/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:55.558457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98abc4cf-4f77-46b7-bf80-f24a9842b398 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1a70fc4d1514b13af12318ccae4cc9b, user_id=357926d5d84f4c38a57a56b240a5d4ca, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:55.559873 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98abc4cf-4f77-46b7-bf80-f24a9842b398 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:55.573120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 580/1153] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:04:55 2022] PUT /identity/v3/projects/c1a70fc4d1514b13af12318ccae4cc9b/users/357926d5d84f4c38a57a56b240a5d4ca/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:04:57.552451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02840b93-350e-46a0-bde2-1f6e4b77f849 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:57.553019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02840b93-350e-46a0-bde2-1f6e4b77f849 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:57.553019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02840b93-350e-46a0-bde2-1f6e4b77f849 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:57.554851 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 574/1154] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:57.560414 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89661885-067b-4622-84a9-052ea2ebe073 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:57.560869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89661885-067b-4622-84a9-052ea2ebe073 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:57.560963 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89661885-067b-4622-84a9-052ea2ebe073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:57.580898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-89661885-067b-4622-84a9-052ea2ebe073 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:57.582087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-89661885-067b-4622-84a9-052ea2ebe073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:57.606741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:57.650922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 581/1155] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:04:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:04:57.934315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02840b93-350e-46a0-bde2-1f6e4b77f849 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:57.946367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:58.037482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02840b93-350e-46a0-bde2-1f6e4b77f849 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:58.070840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a85657-a297-45fc-9d34-e0280efa3253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:58.071463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a85657-a297-45fc-9d34-e0280efa3253 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:58.071740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a85657-a297-45fc-9d34-e0280efa3253 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:58.072034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a85657-a297-45fc-9d34-e0280efa3253 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:58.082406 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a85657-a297-45fc-9d34-e0280efa3253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-f6a85657-a297-45fc-9d34-e0280efa3253 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:04:58.088046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:04:58.089271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:04:58.090135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 575/1156] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:04:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:04:58.095263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89661885-067b-4622-84a9-052ea2ebe073 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:58.199010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89661885-067b-4622-84a9-052ea2ebe073 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:58.234399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb73434-25d5-457f-9779-357ad174d7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:58.235021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb73434-25d5-457f-9779-357ad174d7ad None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:58.235298 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb73434-25d5-457f-9779-357ad174d7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:58.235604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb73434-25d5-457f-9779-357ad174d7ad None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:58.236439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb73434-25d5-457f-9779-357ad174d7ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:58.236762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb73434-25d5-457f-9779-357ad174d7ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:58.238135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb73434-25d5-457f-9779-357ad174d7ad None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:58.247258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 582/1157] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:04:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:58.253307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a85657-a297-45fc-9d34-e0280efa3253 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:58.322257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:58.355790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a85657-a297-45fc-9d34-e0280efa3253 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:58.389051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83f0c1-9a4f-49ad-87f2-40c44bea5ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:58.389670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83f0c1-9a4f-49ad-87f2-40c44bea5ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:58.389945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83f0c1-9a4f-49ad-87f2-40c44bea5ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:58.390210 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83f0c1-9a4f-49ad-87f2-40c44bea5ef0 None admin] PATH_INFO: `/v3/users/357926d5d84f4c38a57a56b240a5d4ca` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:58.391307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83f0c1-9a4f-49ad-87f2-40c44bea5ef0 None admin] RBAC: Authorizing `identity:get_user(user_id=357926d5d84f4c38a57a56b240a5d4ca)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:58.392760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83f0c1-9a4f-49ad-87f2-40c44bea5ef0 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:58.394331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 576/1158] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:04:58 2022] GET /identity/v3/users/357926d5d84f4c38a57a56b240a5d4ca => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:58.400601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb73434-25d5-457f-9779-357ad174d7ad None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:58.426208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:58.506574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb73434-25d5-457f-9779-357ad174d7ad None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:58.541376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3ced7-b5fa-4d3a-ab75-1f670b8bd694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:58.541981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3ced7-b5fa-4d3a-ab75-1f670b8bd694 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:58.542251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3ced7-b5fa-4d3a-ab75-1f670b8bd694 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:58.542520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3ced7-b5fa-4d3a-ab75-1f670b8bd694 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:58.613303 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3ced7-b5fa-4d3a-ab75-1f670b8bd694 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:58.615302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3ced7-b5fa-4d3a-ab75-1f670b8bd694 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:58.642004 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:58.729797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 583/1159] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:04:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:04:58.736282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83f0c1-9a4f-49ad-87f2-40c44bea5ef0 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:58.855872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83f0c1-9a4f-49ad-87f2-40c44bea5ef0 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:58.890527 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8bf69-60df-42b4-a0cb-62888534dc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:58.891168 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8bf69-60df-42b4-a0cb-62888534dc25 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:58.891449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8bf69-60df-42b4-a0cb-62888534dc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:58.891725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8bf69-60df-42b4-a0cb-62888534dc25 None admin] PATH_INFO: `/v3/projects/c1a70fc4d1514b13af12318ccae4cc9b` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:58.892876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8bf69-60df-42b4-a0cb-62888534dc25 None admin] RBAC: Authorizing `identity:get_project(project_id=c1a70fc4d1514b13af12318ccae4cc9b)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:58.894314 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8bf69-60df-42b4-a0cb-62888534dc25 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:58.895969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 577/1160] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:04:58 2022] GET /identity/v3/projects/c1a70fc4d1514b13af12318ccae4cc9b => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:04:58.902843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3ced7-b5fa-4d3a-ab75-1f670b8bd694 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:59.006269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3ced7-b5fa-4d3a-ab75-1f670b8bd694 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:59.033229 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:59.049978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c64754-9fa9-4c81-b564-15cd8a13b65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:59.050599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c64754-9fa9-4c81-b564-15cd8a13b65a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:59.050878 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c64754-9fa9-4c81-b564-15cd8a13b65a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:59.051149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c64754-9fa9-4c81-b564-15cd8a13b65a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:59.051893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c64754-9fa9-4c81-b564-15cd8a13b65a None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=c1a70fc4d1514b13af12318ccae4cc9b, user.id=357926d5d84f4c38a57a56b240a5d4ca) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:04:59.052235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c64754-9fa9-4c81-b564-15cd8a13b65a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:59.053686 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c64754-9fa9-4c81-b564-15cd8a13b65a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:59.060922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 584/1161] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:04:58 2022] GET /identity/v3/role_assignments?user.id=357926d5d84f4c38a57a56b240a5d4ca&scope.project.id=c1a70fc4d1514b13af12318ccae4cc9b&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:01.031667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bbb37-ee13-4bc5-a97c-9c893bb94283 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:01.032206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bbb37-ee13-4bc5-a97c-9c893bb94283 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:01.032293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bbb37-ee13-4bc5-a97c-9c893bb94283 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:01.033623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 578/1162] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:01.039525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db1f6a-5318-4f9f-97ee-330c2dfcc724 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:01.040381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db1f6a-5318-4f9f-97ee-330c2dfcc724 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:01.040381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db1f6a-5318-4f9f-97ee-330c2dfcc724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:01.060552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-21db1f6a-5318-4f9f-97ee-330c2dfcc724 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:01.061759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-21db1f6a-5318-4f9f-97ee-330c2dfcc724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:01.135159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 585/1163] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:05:01.410451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bbb37-ee13-4bc5-a97c-9c893bb94283 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:01.415065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:01.519562 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bbb37-ee13-4bc5-a97c-9c893bb94283 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:01.557560 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca07370-88cf-4d37-a47a-14ab5a77e721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:01.558166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca07370-88cf-4d37-a47a-14ab5a77e721 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:01.558455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca07370-88cf-4d37-a47a-14ab5a77e721 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:01.558726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca07370-88cf-4d37-a47a-14ab5a77e721 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:01.632978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca07370-88cf-4d37-a47a-14ab5a77e721 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:01.634527 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca07370-88cf-4d37-a47a-14ab5a77e721 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:01.637790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:01.747765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 579/1164] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:05:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:05:01.753834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db1f6a-5318-4f9f-97ee-330c2dfcc724 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:01.859493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db1f6a-5318-4f9f-97ee-330c2dfcc724 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:01.876806 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:01.894387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7bf45-9604-4280-bf2c-5730f7d2fbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:01.895014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7bf45-9604-4280-bf2c-5730f7d2fbbc None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:01.895284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7bf45-9604-4280-bf2c-5730f7d2fbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:01.895547 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7bf45-9604-4280-bf2c-5730f7d2fbbc None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:01.903573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7bf45-9604-4280-bf2c-5730f7d2fbbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:01.905082 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7bf45-9604-4280-bf2c-5730f7d2fbbc None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-89b7bf45-9604-4280-bf2c-5730f7d2fbbc None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:05:01.917950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:05:01.919044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:05:01.920098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 586/1165] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Aug 30 01:05:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:05:01.925112 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca07370-88cf-4d37-a47a-14ab5a77e721 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:02.029770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca07370-88cf-4d37-a47a-14ab5a77e721 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:02.032041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:02.066913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27222550-c8ca-4bcb-a917-9ea987b86e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:02.067517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222550-c8ca-4bcb-a917-9ea987b86e87 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:02.067789 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222550-c8ca-4bcb-a917-9ea987b86e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:02.068092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222550-c8ca-4bcb-a917-9ea987b86e87 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:02.068733 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222550-c8ca-4bcb-a917-9ea987b86e87 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:05:02.069032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222550-c8ca-4bcb-a917-9ea987b86e87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:02.070456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222550-c8ca-4bcb-a917-9ea987b86e87 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:02.078705 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 580/1166] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Aug 30 01:05:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:04.015107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ee9d7-4997-4977-a669-784f397a8b24 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:04.015666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ee9d7-4997-4977-a669-784f397a8b24 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:04.015666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ee9d7-4997-4977-a669-784f397a8b24 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:04.016933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 587/1167] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:04.023147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cb0ec-bf5c-49cc-9305-7d9bca5568ff None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:04.023702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cb0ec-bf5c-49cc-9305-7d9bca5568ff None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:04.023702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cb0ec-bf5c-49cc-9305-7d9bca5568ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:04.041868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-a12cb0ec-bf5c-49cc-9305-7d9bca5568ff None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:04.043047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a12cb0ec-bf5c-49cc-9305-7d9bca5568ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:04.111817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 581/1168] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:05:04.384864 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ee9d7-4997-4977-a669-784f397a8b24 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:04.493692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ee9d7-4997-4977-a669-784f397a8b24 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:04.532885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8d587-c1d0-47be-ad49-b8730d471700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:04.533500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8d587-c1d0-47be-ad49-b8730d471700 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:04.533774 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8d587-c1d0-47be-ad49-b8730d471700 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:04.534039 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8d587-c1d0-47be-ad49-b8730d471700 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:04.534726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8d587-c1d0-47be-ad49-b8730d471700 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:04.536151 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8d587-c1d0-47be-ad49-b8730d471700 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:04.554269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 588/1169] 10.222.0.44 () {60 vars in 1138 bytes} [Tue Aug 30 01:05:04 2022] POST /identity/v3/domains => generated 267 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:05:06.517414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10adff-d780-4253-b1d9-a1404cc0b519 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:06.517959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10adff-d780-4253-b1d9-a1404cc0b519 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:06.517959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10adff-d780-4253-b1d9-a1404cc0b519 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:06.519294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 582/1170] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:06.525377 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d38a9-2e9f-4615-84b9-7fe970cf547a None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:06.525835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d38a9-2e9f-4615-84b9-7fe970cf547a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:06.525949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d38a9-2e9f-4615-84b9-7fe970cf547a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:06.544913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-df2d38a9-2e9f-4615-84b9-7fe970cf547a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:06.546097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-df2d38a9-2e9f-4615-84b9-7fe970cf547a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:06.575254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:06.632648 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 589/1171] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:05:06.914836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10adff-d780-4253-b1d9-a1404cc0b519 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:07.017761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:07.023730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10adff-d780-4253-b1d9-a1404cc0b519 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:07.058932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da36a8-53c4-4ee2-8652-beebb0b8c808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:07.059561 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da36a8-53c4-4ee2-8652-beebb0b8c808 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:07.059839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da36a8-53c4-4ee2-8652-beebb0b8c808 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:07.060145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da36a8-53c4-4ee2-8652-beebb0b8c808 None admin] PATH_INFO: `/v3/domains/a6811ff5b33f4b20bceaf7bc7fe976f6` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:07.072386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da36a8-53c4-4ee2-8652-beebb0b8c808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a6811ff5b33f4b20bceaf7bc7fe976f6)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:07.073843 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da36a8-53c4-4ee2-8652-beebb0b8c808 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:07.075531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 583/1172] 10.222.0.44 () {56 vars in 1183 bytes} [Tue Aug 30 01:05:06 2022] GET /identity/v3/domains/a6811ff5b33f4b20bceaf7bc7fe976f6 => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:07.081504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d38a9-2e9f-4615-84b9-7fe970cf547a None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:07.189073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d38a9-2e9f-4615-84b9-7fe970cf547a None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:07.233383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e554933-0f05-4355-a0eb-9913d0781010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:07.233992 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554933-0f05-4355-a0eb-9913d0781010 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:07.234272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554933-0f05-4355-a0eb-9913d0781010 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:07.234537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e554933-0f05-4355-a0eb-9913d0781010 None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:07.235214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554933-0f05-4355-a0eb-9913d0781010 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:07.236664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e554933-0f05-4355-a0eb-9913d0781010 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:07.266210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 590/1173] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:05:07 2022] POST /identity/v3/projects => generated 361 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:05:09.319511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be419b4-7f67-4dc7-8176-5a76156c73e6 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:09.320295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be419b4-7f67-4dc7-8176-5a76156c73e6 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:09.320295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be419b4-7f67-4dc7-8176-5a76156c73e6 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:09.321461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 584/1174] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:09.326852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfd5de-b2c5-4d04-8daa-b3dd84405242 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:09.327272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfd5de-b2c5-4d04-8daa-b3dd84405242 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:09.327405 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfd5de-b2c5-4d04-8daa-b3dd84405242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:09.346306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-99bfd5de-b2c5-4d04-8daa-b3dd84405242 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:09.347535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-99bfd5de-b2c5-4d04-8daa-b3dd84405242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:09.362239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:09.419344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 591/1175] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:05:09.695579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be419b4-7f67-4dc7-8176-5a76156c73e6 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:09.797546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be419b4-7f67-4dc7-8176-5a76156c73e6 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:09.830889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885cdd2-89c9-43de-bcf0-8350cb85b679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:09.831514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885cdd2-89c9-43de-bcf0-8350cb85b679 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:09.831791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885cdd2-89c9-43de-bcf0-8350cb85b679 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:09.832088 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885cdd2-89c9-43de-bcf0-8350cb85b679 None admin] PATH_INFO: `/v3/domains/a6811ff5b33f4b20bceaf7bc7fe976f6` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:09.833190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885cdd2-89c9-43de-bcf0-8350cb85b679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a6811ff5b33f4b20bceaf7bc7fe976f6)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:09.834759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885cdd2-89c9-43de-bcf0-8350cb85b679 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:09.836867 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 585/1176] 10.222.0.44 () {56 vars in 1183 bytes} [Tue Aug 30 01:05:09 2022] GET /identity/v3/domains/a6811ff5b33f4b20bceaf7bc7fe976f6 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:09.842593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfd5de-b2c5-4d04-8daa-b3dd84405242 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:09.940578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfd5de-b2c5-4d04-8daa-b3dd84405242 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:09.974658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a75fce-e077-48f6-b971-96bddbe62aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:09.975159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a75fce-e077-48f6-b971-96bddbe62aa7 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:09.975428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a75fce-e077-48f6-b971-96bddbe62aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:09.975689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a75fce-e077-48f6-b971-96bddbe62aa7 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:09.976371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a75fce-e077-48f6-b971-96bddbe62aa7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:09.977778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a75fce-e077-48f6-b971-96bddbe62aa7 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:09.999162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 592/1177] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:05:09 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:05:11.973032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449db8e-1f14-42c5-ab79-254476374463 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:11.973536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449db8e-1f14-42c5-ab79-254476374463 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:11.973536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449db8e-1f14-42c5-ab79-254476374463 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:11.974958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 586/1178] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:11.980322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad518e-02c0-4021-8f0c-e994ce00aafd None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:11.980716 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad518e-02c0-4021-8f0c-e994ce00aafd None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:11.980876 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad518e-02c0-4021-8f0c-e994ce00aafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:11.988153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:12.000010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-12ad518e-02c0-4021-8f0c-e994ce00aafd None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:12.001192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-12ad518e-02c0-4021-8f0c-e994ce00aafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:12.069288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 593/1179] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:05:12.339051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449db8e-1f14-42c5-ab79-254476374463 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:12.368529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:12.458013 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449db8e-1f14-42c5-ab79-254476374463 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:12.514807 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d5711-7129-4cbe-bb5e-996bbd6eef7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:12.515963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d5711-7129-4cbe-bb5e-996bbd6eef7e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:12.516469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d5711-7129-4cbe-bb5e-996bbd6eef7e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:12.516957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d5711-7129-4cbe-bb5e-996bbd6eef7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:12.539323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d5711-7129-4cbe-bb5e-996bbd6eef7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-642d5711-7129-4cbe-bb5e-996bbd6eef7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:05:12.548680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:05:12.549578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:05:12.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:05:12.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:05:12.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:05:12.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:05:12.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:05:12.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:05:12.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:05:12.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:05:12.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:05:12.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:05:12.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:05:12.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:05:12.551134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 587/1180] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:05:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:05:12.557224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad518e-02c0-4021-8f0c-e994ce00aafd None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:12.653350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad518e-02c0-4021-8f0c-e994ce00aafd None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:12.687797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fa2bb-cba8-4c4b-bab7-07a9128031ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:12.688441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fa2bb-cba8-4c4b-bab7-07a9128031ed None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:12.688720 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fa2bb-cba8-4c4b-bab7-07a9128031ed None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:12.688991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fa2bb-cba8-4c4b-bab7-07a9128031ed None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:12.689658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fa2bb-cba8-4c4b-bab7-07a9128031ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:05:12.689960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fa2bb-cba8-4c4b-bab7-07a9128031ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:12.691332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fa2bb-cba8-4c4b-bab7-07a9128031ed None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:12.692801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:12.701373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 594/1181] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:05:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:12.707511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d5711-7129-4cbe-bb5e-996bbd6eef7e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:12.792414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:12.816291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d5711-7129-4cbe-bb5e-996bbd6eef7e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:12.864631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b321be-14a9-4908-abe8-1a4c887b6a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:12.866147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b321be-14a9-4908-abe8-1a4c887b6a79 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:12.866642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b321be-14a9-4908-abe8-1a4c887b6a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:12.867082 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b321be-14a9-4908-abe8-1a4c887b6a79 None admin] PATH_INFO: `/v3/users/603c12a3fca24c11821ac0a3ad904a3d` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:12.881629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b321be-14a9-4908-abe8-1a4c887b6a79 None admin] RBAC: Authorizing `identity:get_user(user_id=603c12a3fca24c11821ac0a3ad904a3d)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:12.883229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b321be-14a9-4908-abe8-1a4c887b6a79 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:12.885044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 588/1182] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:05:12 2022] GET /identity/v3/users/603c12a3fca24c11821ac0a3ad904a3d => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:12.891159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fa2bb-cba8-4c4b-bab7-07a9128031ed None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:12.997265 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fa2bb-cba8-4c4b-bab7-07a9128031ed None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:13.033273 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9e059-5f3a-47cd-8a53-9a7829dd4d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:13.033910 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e059-5f3a-47cd-8a53-9a7829dd4d21 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:13.034222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e059-5f3a-47cd-8a53-9a7829dd4d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:13.034494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e059-5f3a-47cd-8a53-9a7829dd4d21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:13.098428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:13.105144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9e059-5f3a-47cd-8a53-9a7829dd4d21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:13.106870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9e059-5f3a-47cd-8a53-9a7829dd4d21 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:13.215044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 595/1183] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:05:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:05:13.221856 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b321be-14a9-4908-abe8-1a4c887b6a79 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:13.328035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b321be-14a9-4908-abe8-1a4c887b6a79 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:13.364639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565b8b9-6922-47ea-b2cf-74a53dbda61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:13.365915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565b8b9-6922-47ea-b2cf-74a53dbda61e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:13.366195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565b8b9-6922-47ea-b2cf-74a53dbda61e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:13.366703 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565b8b9-6922-47ea-b2cf-74a53dbda61e None admin] PATH_INFO: `/v3/projects/3cea5f9a9a06459ea541d23aecebac66` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:13.367876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565b8b9-6922-47ea-b2cf-74a53dbda61e None admin] RBAC: Authorizing `identity:get_project(project_id=3cea5f9a9a06459ea541d23aecebac66)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:13.369410 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565b8b9-6922-47ea-b2cf-74a53dbda61e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:13.371143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 589/1184] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:05:13 2022] GET /identity/v3/projects/3cea5f9a9a06459ea541d23aecebac66 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:13.377783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9e059-5f3a-47cd-8a53-9a7829dd4d21 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:13.478465 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:13.484901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9e059-5f3a-47cd-8a53-9a7829dd4d21 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:13.520181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815c46f-dac9-4492-9ddd-152dd1870263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:13.520788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815c46f-dac9-4492-9ddd-152dd1870263 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:13.521064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815c46f-dac9-4492-9ddd-152dd1870263 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:13.521371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815c46f-dac9-4492-9ddd-152dd1870263 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:13.522207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815c46f-dac9-4492-9ddd-152dd1870263 None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=3cea5f9a9a06459ea541d23aecebac66, user.id=603c12a3fca24c11821ac0a3ad904a3d) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:05:13.522510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815c46f-dac9-4492-9ddd-152dd1870263 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:13.524029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815c46f-dac9-4492-9ddd-152dd1870263 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:13.530448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 596/1185] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:05:13 2022] GET /identity/v3/role_assignments?user.id=603c12a3fca24c11821ac0a3ad904a3d&scope.project.id=3cea5f9a9a06459ea541d23aecebac66&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:15.499133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a6458-c346-42c0-ae26-fbb1a969a519 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:15.499663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a6458-c346-42c0-ae26-fbb1a969a519 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:15.499663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a6458-c346-42c0-ae26-fbb1a969a519 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:15.501053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 590/1186] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:15.506291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a6f94-a160-455e-b952-6093156c64c9 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:15.506689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a6f94-a160-455e-b952-6093156c64c9 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:15.506884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a6f94-a160-455e-b952-6093156c64c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:15.525613 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ab3a6f94-a160-455e-b952-6093156c64c9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:15.526892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ab3a6f94-a160-455e-b952-6093156c64c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:15.594976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 597/1187] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:05:15.865041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a6458-c346-42c0-ae26-fbb1a969a519 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:15.879651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:15.966268 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a6458-c346-42c0-ae26-fbb1a969a519 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:16.000181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d04ac-2edd-4766-ab77-479adb074050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:16.000799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d04ac-2edd-4766-ab77-479adb074050 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:16.001073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d04ac-2edd-4766-ab77-479adb074050 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:16.001338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d04ac-2edd-4766-ab77-479adb074050 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:16.012814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d04ac-2edd-4766-ab77-479adb074050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-864d04ac-2edd-4766-ab77-479adb074050 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:05:16.017680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:05:16.018786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:05:16.019695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 591/1188] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:05:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:05:16.026312 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a6f94-a160-455e-b952-6093156c64c9 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:16.116832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:16.126746 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a6f94-a160-455e-b952-6093156c64c9 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:16.165394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7391f-a3c9-4e44-98ae-7df6eb4d2ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:16.166032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7391f-a3c9-4e44-98ae-7df6eb4d2ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:16.166306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7391f-a3c9-4e44-98ae-7df6eb4d2ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:16.166574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7391f-a3c9-4e44-98ae-7df6eb4d2ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:16.167239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7391f-a3c9-4e44-98ae-7df6eb4d2ee7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:05:16.167550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7391f-a3c9-4e44-98ae-7df6eb4d2ee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:16.168955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7391f-a3c9-4e44-98ae-7df6eb4d2ee7 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:16.178356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 598/1189] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:05:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:16.185106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d04ac-2edd-4766-ab77-479adb074050 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:16.258690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:16.288851 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d04ac-2edd-4766-ab77-479adb074050 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:16.324597 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd444f-c17f-4d4d-881f-979e0c5a8acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:16.325258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd444f-c17f-4d4d-881f-979e0c5a8acf None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:16.325535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd444f-c17f-4d4d-881f-979e0c5a8acf None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:16.325804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd444f-c17f-4d4d-881f-979e0c5a8acf None admin] PATH_INFO: `/v3/users/603c12a3fca24c11821ac0a3ad904a3d` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:16.326916 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd444f-c17f-4d4d-881f-979e0c5a8acf None admin] RBAC: Authorizing `identity:get_user(user_id=603c12a3fca24c11821ac0a3ad904a3d)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:16.328371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd444f-c17f-4d4d-881f-979e0c5a8acf None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:16.329961 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 592/1190] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:05:16 2022] GET /identity/v3/users/603c12a3fca24c11821ac0a3ad904a3d => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:16.335788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7391f-a3c9-4e44-98ae-7df6eb4d2ee7 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:16.434688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7391f-a3c9-4e44-98ae-7df6eb4d2ee7 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:16.468515 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b288dfc-5c06-4ad9-a59a-d9cd2270073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:16.469145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b288dfc-5c06-4ad9-a59a-d9cd2270073d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:16.469427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b288dfc-5c06-4ad9-a59a-d9cd2270073d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:16.469694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b288dfc-5c06-4ad9-a59a-d9cd2270073d None admin] PATH_INFO: `/v3/projects/3cea5f9a9a06459ea541d23aecebac66` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:16.470805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b288dfc-5c06-4ad9-a59a-d9cd2270073d None admin] RBAC: Authorizing `identity:get_project(project_id=3cea5f9a9a06459ea541d23aecebac66)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:16.472275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b288dfc-5c06-4ad9-a59a-d9cd2270073d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:16.473937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 599/1191] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:05:16 2022] GET /identity/v3/projects/3cea5f9a9a06459ea541d23aecebac66 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:16.480894 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd444f-c17f-4d4d-881f-979e0c5a8acf None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:16.585856 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd444f-c17f-4d4d-881f-979e0c5a8acf None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:16.620340 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88224161-7084-409a-901b-424c1fa6eca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:16.621033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88224161-7084-409a-901b-424c1fa6eca5 None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:16.621277 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88224161-7084-409a-901b-424c1fa6eca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:16.621547 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88224161-7084-409a-901b-424c1fa6eca5 None admin] PATH_INFO: `/v3/projects/3cea5f9a9a06459ea541d23aecebac66/users/603c12a3fca24c11821ac0a3ad904a3d/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:16.623459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88224161-7084-409a-901b-424c1fa6eca5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3cea5f9a9a06459ea541d23aecebac66, user_id=603c12a3fca24c11821ac0a3ad904a3d, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:16.624915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88224161-7084-409a-901b-424c1fa6eca5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:16.633518 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:16.638363 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 593/1192] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:05:16 2022] PUT /identity/v3/projects/3cea5f9a9a06459ea541d23aecebac66/users/603c12a3fca24c11821ac0a3ad904a3d/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:05:18.574167 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69f31a-0c00-4c07-8cea-45e963aa6066 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:18.574834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69f31a-0c00-4c07-8cea-45e963aa6066 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:18.574834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69f31a-0c00-4c07-8cea-45e963aa6066 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:18.576469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 600/1193] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:18.582213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5e7ab-558c-46e8-83e8-3661ad715093 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:18.582863 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5e7ab-558c-46e8-83e8-3661ad715093 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:18.582863 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5e7ab-558c-46e8-83e8-3661ad715093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:18.603493 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-efb5e7ab-558c-46e8-83e8-3661ad715093 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:18.604749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-efb5e7ab-558c-46e8-83e8-3661ad715093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:18.674349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 594/1194] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:05:18.950355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69f31a-0c00-4c07-8cea-45e963aa6066 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:18.973169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:19.078295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69f31a-0c00-4c07-8cea-45e963aa6066 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:19.113364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251cdde-da83-45be-a891-9090b8d3c6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:19.113996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251cdde-da83-45be-a891-9090b8d3c6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:19.114274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251cdde-da83-45be-a891-9090b8d3c6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:19.114744 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251cdde-da83-45be-a891-9090b8d3c6b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:19.125186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251cdde-da83-45be-a891-9090b8d3c6b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-1251cdde-da83-45be-a891-9090b8d3c6b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:05:19.129986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:05:19.131211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:05:19.132358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 601/1195] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:05:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:05:19.137959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5e7ab-558c-46e8-83e8-3661ad715093 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:19.246777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5e7ab-558c-46e8-83e8-3661ad715093 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:19.276129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:19.282461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcb756-1260-45d1-92a5-cc99516c276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:19.283075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcb756-1260-45d1-92a5-cc99516c276e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:19.283385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcb756-1260-45d1-92a5-cc99516c276e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:19.283655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcb756-1260-45d1-92a5-cc99516c276e None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:19.284361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcb756-1260-45d1-92a5-cc99516c276e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:05:19.284666 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcb756-1260-45d1-92a5-cc99516c276e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:19.286057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcb756-1260-45d1-92a5-cc99516c276e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:19.295096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 595/1196] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:05:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:19.301271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251cdde-da83-45be-a891-9090b8d3c6b6 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:19.385907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:19.406631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251cdde-da83-45be-a891-9090b8d3c6b6 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:19.443041 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cf5d7-e22f-4818-9338-437ee6f38ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:19.443755 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cf5d7-e22f-4818-9338-437ee6f38ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:19.444057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cf5d7-e22f-4818-9338-437ee6f38ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:19.444330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cf5d7-e22f-4818-9338-437ee6f38ac5 None admin] PATH_INFO: `/v3/users/603c12a3fca24c11821ac0a3ad904a3d` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:19.445469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cf5d7-e22f-4818-9338-437ee6f38ac5 None admin] RBAC: Authorizing `identity:get_user(user_id=603c12a3fca24c11821ac0a3ad904a3d)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:19.446911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cf5d7-e22f-4818-9338-437ee6f38ac5 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:19.448645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 602/1197] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Aug 30 01:05:19 2022] GET /identity/v3/users/603c12a3fca24c11821ac0a3ad904a3d => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:19.455002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcb756-1260-45d1-92a5-cc99516c276e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:19.562802 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcb756-1260-45d1-92a5-cc99516c276e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:19.599442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62afbf-8ba0-40de-9544-c48a86cbdf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:19.600539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62afbf-8ba0-40de-9544-c48a86cbdf72 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:19.600820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62afbf-8ba0-40de-9544-c48a86cbdf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:19.601092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62afbf-8ba0-40de-9544-c48a86cbdf72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:19.665465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:19.675617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62afbf-8ba0-40de-9544-c48a86cbdf72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:19.677190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62afbf-8ba0-40de-9544-c48a86cbdf72 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:19.791916 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 596/1198] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:05:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:05:19.798204 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cf5d7-e22f-4818-9338-437ee6f38ac5 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:19.902725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cf5d7-e22f-4818-9338-437ee6f38ac5 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:19.938312 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9324894-8e92-4d67-bc8c-d969e45e5cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:19.938937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9324894-8e92-4d67-bc8c-d969e45e5cfa None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:19.939208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9324894-8e92-4d67-bc8c-d969e45e5cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:19.939474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9324894-8e92-4d67-bc8c-d969e45e5cfa None admin] PATH_INFO: `/v3/projects/3cea5f9a9a06459ea541d23aecebac66` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:19.940595 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9324894-8e92-4d67-bc8c-d969e45e5cfa None admin] RBAC: Authorizing `identity:get_project(project_id=3cea5f9a9a06459ea541d23aecebac66)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:19.942031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9324894-8e92-4d67-bc8c-d969e45e5cfa None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:19.943708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 603/1199] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Aug 30 01:05:19 2022] GET /identity/v3/projects/3cea5f9a9a06459ea541d23aecebac66 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:19.950074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62afbf-8ba0-40de-9544-c48a86cbdf72 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:20.057797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:20.067184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62afbf-8ba0-40de-9544-c48a86cbdf72 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:20.101980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa577518-ea2d-403f-a285-eaba4a335510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:20.102588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa577518-ea2d-403f-a285-eaba4a335510 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:20.102859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa577518-ea2d-403f-a285-eaba4a335510 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:20.103125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa577518-ea2d-403f-a285-eaba4a335510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:20.103841 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa577518-ea2d-403f-a285-eaba4a335510 None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=3cea5f9a9a06459ea541d23aecebac66, user.id=603c12a3fca24c11821ac0a3ad904a3d) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:05:20.104174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa577518-ea2d-403f-a285-eaba4a335510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:20.105586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa577518-ea2d-403f-a285-eaba4a335510 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:20.111572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 597/1200] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:05:19 2022] GET /identity/v3/role_assignments?user.id=603c12a3fca24c11821ac0a3ad904a3d&scope.project.id=3cea5f9a9a06459ea541d23aecebac66&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:05:29.933447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=92 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:30.108947 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=22 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:19.402418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfa912-127c-49e7-92d2-685db9d0530b None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:19.404673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfa912-127c-49e7-92d2-685db9d0530b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:19.405468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfa912-127c-49e7-92d2-685db9d0530b None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:19.409212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 604/1201] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:06:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:06:19.415295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03c31d-7c1c-4664-bbf1-7cc4a2992c33 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:19.416069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03c31d-7c1c-4664-bbf1-7cc4a2992c33 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:19.416069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03c31d-7c1c-4664-bbf1-7cc4a2992c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:19.455966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8e03c31d-7c1c-4664-bbf1-7cc4a2992c33 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:19.459059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8e03c31d-7c1c-4664-bbf1-7cc4a2992c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:19.611507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 598/1202] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:06:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:06:19.981747 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dfa912-127c-49e7-92d2-685db9d0530b None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:20.192330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dfa912-127c-49e7-92d2-685db9d0530b None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:20.262015 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0f16a-d931-4c04-b3f7-45302dda9d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:20.262591 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0f16a-d931-4c04-b3f7-45302dda9d1d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:20.262865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0f16a-d931-4c04-b3f7-45302dda9d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:20.263133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0f16a-d931-4c04-b3f7-45302dda9d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:20.287343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0f16a-d931-4c04-b3f7-45302dda9d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:20.290811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0f16a-d931-4c04-b3f7-45302dda9d1d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-87d0f16a-d931-4c04-b3f7-45302dda9d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:20.327470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:20.329551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:20.331451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:20.331451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:20.331451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:20.331451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 605/1203] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:20.342616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03c31d-7c1c-4664-bbf1-7cc4a2992c33 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:20.579013 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03c31d-7c1c-4664-bbf1-7cc4a2992c33 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:20.627558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:20.642775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea91cb8-aee8-4180-bc24-a177f583f486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:20.643434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea91cb8-aee8-4180-bc24-a177f583f486 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:20.643666 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea91cb8-aee8-4180-bc24-a177f583f486 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:20.644032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea91cb8-aee8-4180-bc24-a177f583f486 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:20.644782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea91cb8-aee8-4180-bc24-a177f583f486 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:20.645151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea91cb8-aee8-4180-bc24-a177f583f486 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:20.646785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea91cb8-aee8-4180-bc24-a177f583f486 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:20.663308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 599/1204] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:06:20.669644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0f16a-d931-4c04-b3f7-45302dda9d1d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:20.753134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:20.820216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0f16a-d931-4c04-b3f7-45302dda9d1d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:20.870583 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb276cc-91bc-4565-a1e9-1468fb165d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:20.871706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb276cc-91bc-4565-a1e9-1468fb165d3e None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:20.872283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb276cc-91bc-4565-a1e9-1468fb165d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:20.872775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb276cc-91bc-4565-a1e9-1468fb165d3e None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:20.874488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb276cc-91bc-4565-a1e9-1468fb165d3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:20.877663 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb276cc-91bc-4565-a1e9-1468fb165d3e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:20.922919 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 606/1205] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Aug 30 01:06:20 2022] POST /identity/v3/users => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:06:23.384028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc951f-47e7-4a04-9e79-a0ec7750be9b None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:23.384681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc951f-47e7-4a04-9e79-a0ec7750be9b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:23.384681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc951f-47e7-4a04-9e79-a0ec7750be9b None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:23.386226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 600/1206] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:06:23.392109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16c3a4-ae66-4ff0-8a66-64bbe67ffa17 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:23.392902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16c3a4-ae66-4ff0-8a66-64bbe67ffa17 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:23.392902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16c3a4-ae66-4ff0-8a66-64bbe67ffa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:23.420258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-0c16c3a4-ae66-4ff0-8a66-64bbe67ffa17 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:23.420258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0c16c3a4-ae66-4ff0-8a66-64bbe67ffa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:23.522332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 607/1207] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:06:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:06:23.836859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc951f-47e7-4a04-9e79-a0ec7750be9b None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:24.035724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc951f-47e7-4a04-9e79-a0ec7750be9b None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:24.100896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6c7aa-f35e-4e0b-8f7a-946ed1252b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:24.101593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6c7aa-f35e-4e0b-8f7a-946ed1252b88 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:24.101822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6c7aa-f35e-4e0b-8f7a-946ed1252b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:24.102087 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6c7aa-f35e-4e0b-8f7a-946ed1252b88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:24.114884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6c7aa-f35e-4e0b-8f7a-946ed1252b88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-61a6c7aa-f35e-4e0b-8f7a-946ed1252b88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:06:24.119637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:24.120637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:24.121522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:24.121522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:24.121522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:24.121522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:24.121522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:06:24.121522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:06:24.121522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:06:24.121522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:06:24.121522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:06:24.121522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:06:24.121522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:06:24.121522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:24.126083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 601/1208] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:06:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:06:24.128108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16c3a4-ae66-4ff0-8a66-64bbe67ffa17 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:24.204226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:24.300323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16c3a4-ae66-4ff0-8a66-64bbe67ffa17 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:24.368500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76424933-8314-47a3-801b-6a16392d95ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:24.369519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76424933-8314-47a3-801b-6a16392d95ea None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:24.370039 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76424933-8314-47a3-801b-6a16392d95ea None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:24.370542 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76424933-8314-47a3-801b-6a16392d95ea None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:24.371660 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76424933-8314-47a3-801b-6a16392d95ea None admin] RBAC: Adding query filter params (name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:24.372283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76424933-8314-47a3-801b-6a16392d95ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:24.375011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76424933-8314-47a3-801b-6a16392d95ea None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:24.390105 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 608/1209] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:06:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:24.396588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6c7aa-f35e-4e0b-8f7a-946ed1252b88 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:24.479153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:24.547791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6c7aa-f35e-4e0b-8f7a-946ed1252b88 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:24.615216 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a04e13-a798-4dbd-97eb-c81cacb2dfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:24.616247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a04e13-a798-4dbd-97eb-c81cacb2dfe5 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:24.616792 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a04e13-a798-4dbd-97eb-c81cacb2dfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:24.617286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a04e13-a798-4dbd-97eb-c81cacb2dfe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:24.640194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a04e13-a798-4dbd-97eb-c81cacb2dfe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:24.641766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a04e13-a798-4dbd-97eb-c81cacb2dfe5 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-86a04e13-a798-4dbd-97eb-c81cacb2dfe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:24.659191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:24.660286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:24.661156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:24.661156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:24.661156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:24.661156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 602/1210] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:06:24.666341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76424933-8314-47a3-801b-6a16392d95ea None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:24.818290 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:24.822326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76424933-8314-47a3-801b-6a16392d95ea None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:24.875650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7317c-d7b5-47e0-a0f0-5dbe7cba695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:24.876307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7317c-d7b5-47e0-a0f0-5dbe7cba695f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:24.876597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7317c-d7b5-47e0-a0f0-5dbe7cba695f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:24.876867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7317c-d7b5-47e0-a0f0-5dbe7cba695f None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:24.877539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7317c-d7b5-47e0-a0f0-5dbe7cba695f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:24.877843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7317c-d7b5-47e0-a0f0-5dbe7cba695f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:24.879383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7317c-d7b5-47e0-a0f0-5dbe7cba695f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:24.898845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 609/1211] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:24.901010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a04e13-a798-4dbd-97eb-c81cacb2dfe5 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:25.074968 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a04e13-a798-4dbd-97eb-c81cacb2dfe5 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:25.076715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:25.146404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb551928-9c59-4e92-bbf4-60adc8f35e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:25.147467 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb551928-9c59-4e92-bbf4-60adc8f35e0a None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:25.147986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb551928-9c59-4e92-bbf4-60adc8f35e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:25.148768 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb551928-9c59-4e92-bbf4-60adc8f35e0a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:25.159248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb551928-9c59-4e92-bbf4-60adc8f35e0a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:25.160733 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb551928-9c59-4e92-bbf4-60adc8f35e0a None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-bb551928-9c59-4e92-bbf4-60adc8f35e0a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:06:25.173883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:25.175114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:25.176263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:25.176263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:25.176263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:06:25.176263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:06:25.176263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:06:25.176263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:06:25.176263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:06:25.176263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:06:25.176263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:06:25.176263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:25.180031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 603/1212] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:25.181217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7317c-d7b5-47e0-a0f0-5dbe7cba695f None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:25.326142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7317c-d7b5-47e0-a0f0-5dbe7cba695f None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:25.373457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c9b7d-5d05-4a92-9203-a97d5d20a514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:25.374276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c9b7d-5d05-4a92-9203-a97d5d20a514 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:25.374566 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c9b7d-5d05-4a92-9203-a97d5d20a514 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:25.374838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c9b7d-5d05-4a92-9203-a97d5d20a514 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:25.375615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c9b7d-5d05-4a92-9203-a97d5d20a514 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:25.375924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c9b7d-5d05-4a92-9203-a97d5d20a514 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:25.377435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c9b7d-5d05-4a92-9203-a97d5d20a514 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:25.400116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 610/1213] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Aug 30 01:06:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:25.405849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb551928-9c59-4e92-bbf4-60adc8f35e0a None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:25.548396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb551928-9c59-4e92-bbf4-60adc8f35e0a None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:25.608016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41a358-77bf-4030-845e-472dfa56b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:25.609195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41a358-77bf-4030-845e-472dfa56b93f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:25.609686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41a358-77bf-4030-845e-472dfa56b93f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:25.610134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41a358-77bf-4030-845e-472dfa56b93f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:25.665311 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:25.735630 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41a358-77bf-4030-845e-472dfa56b93f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:25.738283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41a358-77bf-4030-845e-472dfa56b93f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:25.862670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 604/1214] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:06:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:06:25.868584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c9b7d-5d05-4a92-9203-a97d5d20a514 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:25.890243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:26.011134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c9b7d-5d05-4a92-9203-a97d5d20a514 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:26.053018 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7449af8-3e46-462f-83fd-5cdde66e68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:26.053979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7449af8-3e46-462f-83fd-5cdde66e68dd None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:26.054455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7449af8-3e46-462f-83fd-5cdde66e68dd None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:26.054963 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7449af8-3e46-462f-83fd-5cdde66e68dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:26.068245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7449af8-3e46-462f-83fd-5cdde66e68dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:26.070327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7449af8-3e46-462f-83fd-5cdde66e68dd None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-b7449af8-3e46-462f-83fd-5cdde66e68dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:26.089852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:26.091164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:26.092630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:26.092630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:26.092630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:26.095170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 611/1215] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:26.101256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41a358-77bf-4030-845e-472dfa56b93f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:26.182973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:26.233626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41a358-77bf-4030-845e-472dfa56b93f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:26.275164 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db70d2-5903-4f9b-b58f-760d448f3e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:26.275801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db70d2-5903-4f9b-b58f-760d448f3e53 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:26.276111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db70d2-5903-4f9b-b58f-760d448f3e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:26.276944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db70d2-5903-4f9b-b58f-760d448f3e53 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:26.277486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db70d2-5903-4f9b-b58f-760d448f3e53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:26.277804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db70d2-5903-4f9b-b58f-760d448f3e53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:26.280147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db70d2-5903-4f9b-b58f-760d448f3e53 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:26.292815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 605/1216] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:06:26.300121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7449af8-3e46-462f-83fd-5cdde66e68dd None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:26.430660 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:26.489593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7449af8-3e46-462f-83fd-5cdde66e68dd None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:26.537955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83becd-18cd-4bd9-8436-37617a6f9f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:26.538685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83becd-18cd-4bd9-8436-37617a6f9f6e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:26.539019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83becd-18cd-4bd9-8436-37617a6f9f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:26.539295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83becd-18cd-4bd9-8436-37617a6f9f6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:26.550424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83becd-18cd-4bd9-8436-37617a6f9f6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:26.552473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83becd-18cd-4bd9-8436-37617a6f9f6e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-8e83becd-18cd-4bd9-8436-37617a6f9f6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:06:26.567480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:26.569343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:26.570867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:06:26.570867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:06:26.570867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:06:26.570867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:06:26.570867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:06:26.570867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:06:26.570867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:06:26.570867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:26.570867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 612/1217] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:06:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:06:26.579246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db70d2-5903-4f9b-b58f-760d448f3e53 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:26.735298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db70d2-5903-4f9b-b58f-760d448f3e53 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:26.769181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:26.782240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b71c0-3f6c-4166-a3b6-506556b68fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:26.782872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b71c0-3f6c-4166-a3b6-506556b68fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:26.783402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b71c0-3f6c-4166-a3b6-506556b68fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:26.783668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b71c0-3f6c-4166-a3b6-506556b68fc8 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:26.784381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42b71c0-3f6c-4166-a3b6-506556b68fc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:26.784681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42b71c0-3f6c-4166-a3b6-506556b68fc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:26.786381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42b71c0-3f6c-4166-a3b6-506556b68fc8 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:26.804283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 606/1218] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:06:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:26.811177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83becd-18cd-4bd9-8436-37617a6f9f6e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:26.960558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83becd-18cd-4bd9-8436-37617a6f9f6e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:27.005352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:27.012599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f8850-83a8-4279-b8a2-19c06c01b650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:27.013975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f8850-83a8-4279-b8a2-19c06c01b650 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:27.014336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f8850-83a8-4279-b8a2-19c06c01b650 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:27.014671 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f8850-83a8-4279-b8a2-19c06c01b650 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:27.015479 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f8850-83a8-4279-b8a2-19c06c01b650 None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=08f34295f19b4d6ea405f1e8c8190db5) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:27.015781 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f8850-83a8-4279-b8a2-19c06c01b650 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:27.017444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f8850-83a8-4279-b8a2-19c06c01b650 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:27.033502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 613/1219] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:06:26 2022] GET /identity/v3/role_assignments?user.id=08f34295f19b4d6ea405f1e8c8190db5&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:29.117281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd4e7f-915b-4c52-bdab-323314b85a45 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:29.117960 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd4e7f-915b-4c52-bdab-323314b85a45 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:29.118215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd4e7f-915b-4c52-bdab-323314b85a45 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:29.119945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 607/1220] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:06:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:06:29.125635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445c31b-37f7-4a1e-b191-fe7255964467 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:29.126158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445c31b-37f7-4a1e-b191-fe7255964467 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:29.126257 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445c31b-37f7-4a1e-b191-fe7255964467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:29.145901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6445c31b-37f7-4a1e-b191-fe7255964467 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:29.147208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6445c31b-37f7-4a1e-b191-fe7255964467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:29.242863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 614/1221] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:06:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:06:29.546760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd4e7f-915b-4c52-bdab-323314b85a45 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:29.669965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd4e7f-915b-4c52-bdab-323314b85a45 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:29.709376 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e85f1-3ca4-4536-8a8e-03b0a7c2a9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:29.710143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e85f1-3ca4-4536-8a8e-03b0a7c2a9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:29.710425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e85f1-3ca4-4536-8a8e-03b0a7c2a9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:29.710711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e85f1-3ca4-4536-8a8e-03b0a7c2a9b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:29.728280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e85f1-3ca4-4536-8a8e-03b0a7c2a9b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-c19e85f1-3ca4-4536-8a8e-03b0a7c2a9b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:06:29.733866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:29.734977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:29.735857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 608/1222] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:06:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:06:29.741749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445c31b-37f7-4a1e-b191-fe7255964467 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:29.875127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445c31b-37f7-4a1e-b191-fe7255964467 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:29.904893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:29.918565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4664f-3fcd-4f17-8a4e-76099bbb35a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:29.919399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4664f-3fcd-4f17-8a4e-76099bbb35a8 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:29.919691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4664f-3fcd-4f17-8a4e-76099bbb35a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:29.919960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4664f-3fcd-4f17-8a4e-76099bbb35a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:29.920884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4664f-3fcd-4f17-8a4e-76099bbb35a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:29.921209 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4664f-3fcd-4f17-8a4e-76099bbb35a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:29.922841 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4664f-3fcd-4f17-8a4e-76099bbb35a8 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:29.939769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 615/1223] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:06:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:29.941669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e85f1-3ca4-4536-8a8e-03b0a7c2a9b9 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:30.005636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:30.081493 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e85f1-3ca4-4536-8a8e-03b0a7c2a9b9 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:30.128264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b025-8ddd-4c94-97fa-cc43141a75d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:30.129646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b025-8ddd-4c94-97fa-cc43141a75d3 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:30.129646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b025-8ddd-4c94-97fa-cc43141a75d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:30.129646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b025-8ddd-4c94-97fa-cc43141a75d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:30.140496 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4b025-8ddd-4c94-97fa-cc43141a75d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:30.141976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4b025-8ddd-4c94-97fa-cc43141a75d3 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-fde4b025-8ddd-4c94-97fa-cc43141a75d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:30.158200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:30.159365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:30.160828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:30.160828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:30.160828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:30.160828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 609/1224] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:30.165478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4664f-3fcd-4f17-8a4e-76099bbb35a8 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:30.287329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4664f-3fcd-4f17-8a4e-76099bbb35a8 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:30.338883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0c183-b3e4-4212-bd43-0cd1f6c25a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:30.339895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0c183-b3e4-4212-bd43-0cd1f6c25a6a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:30.340402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0c183-b3e4-4212-bd43-0cd1f6c25a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:30.340865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0c183-b3e4-4212-bd43-0cd1f6c25a6a None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:30.341941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0c183-b3e4-4212-bd43-0cd1f6c25a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:30.342368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0c183-b3e4-4212-bd43-0cd1f6c25a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:30.344294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0c183-b3e4-4212-bd43-0cd1f6c25a6a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:30.359887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 616/1225] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:30.365742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b025-8ddd-4c94-97fa-cc43141a75d3 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:30.482236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:30.492953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b025-8ddd-4c94-97fa-cc43141a75d3 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:30.535777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de594a-69cc-42a0-81ed-1170e553dcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:30.536621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de594a-69cc-42a0-81ed-1170e553dcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:30.536957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de594a-69cc-42a0-81ed-1170e553dcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:30.537297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de594a-69cc-42a0-81ed-1170e553dcb1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:30.548413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de594a-69cc-42a0-81ed-1170e553dcb1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:30.550265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de594a-69cc-42a0-81ed-1170e553dcb1 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-96de594a-69cc-42a0-81ed-1170e553dcb1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:06:30.567116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:30.568785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:30.570031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:30.570031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:30.570031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:06:30.570031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:06:30.570031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:06:30.570031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:06:30.570031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:06:30.570031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:06:30.570031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:06:30.570031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:30.570031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 610/1226] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:30.574821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0c183-b3e4-4212-bd43-0cd1f6c25a6a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:30.627718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:30.693648 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0c183-b3e4-4212-bd43-0cd1f6c25a6a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:30.733197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968eae3-dbff-464b-8278-452613fd1aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:30.733741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968eae3-dbff-464b-8278-452613fd1aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:30.734043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968eae3-dbff-464b-8278-452613fd1aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:30.734299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968eae3-dbff-464b-8278-452613fd1aa6 None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:30.735137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968eae3-dbff-464b-8278-452613fd1aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:30.735449 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968eae3-dbff-464b-8278-452613fd1aa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:30.736975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968eae3-dbff-464b-8278-452613fd1aa6 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:30.758532 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 617/1227] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Aug 30 01:06:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:30.765468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de594a-69cc-42a0-81ed-1170e553dcb1 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:30.877602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de594a-69cc-42a0-81ed-1170e553dcb1 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:30.920179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660db30-2429-4d3d-a5bf-5ef6c6b743ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:30.920907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660db30-2429-4d3d-a5bf-5ef6c6b743ae None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:30.921269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660db30-2429-4d3d-a5bf-5ef6c6b743ae None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:30.921517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660db30-2429-4d3d-a5bf-5ef6c6b743ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:30.933950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660db30-2429-4d3d-a5bf-5ef6c6b743ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:30.935819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660db30-2429-4d3d-a5bf-5ef6c6b743ae None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-c660db30-2429-4d3d-a5bf-5ef6c6b743ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:30.957984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:30.959658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:30.961171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:30.961171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:30.961171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:30.961171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 611/1228] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:30.965514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968eae3-dbff-464b-8278-452613fd1aa6 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:31.082897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968eae3-dbff-464b-8278-452613fd1aa6 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:31.085622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:31.124517 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb2d88-ace9-4828-a696-8e2b1fc406c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:31.125171 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb2d88-ace9-4828-a696-8e2b1fc406c4 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:31.125437 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb2d88-ace9-4828-a696-8e2b1fc406c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:31.125751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb2d88-ace9-4828-a696-8e2b1fc406c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:31.126536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb2d88-ace9-4828-a696-8e2b1fc406c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:31.126845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb2d88-ace9-4828-a696-8e2b1fc406c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:31.128424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb2d88-ace9-4828-a696-8e2b1fc406c4 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:31.142337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 618/1229] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:31.148395 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660db30-2429-4d3d-a5bf-5ef6c6b743ae None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:31.159313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:31.272322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660db30-2429-4d3d-a5bf-5ef6c6b743ae None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:31.308165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984830af-cfe0-4de2-83b7-d46cf72cf935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:31.308938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984830af-cfe0-4de2-83b7-d46cf72cf935 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:31.309373 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984830af-cfe0-4de2-83b7-d46cf72cf935 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:31.310170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984830af-cfe0-4de2-83b7-d46cf72cf935 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:31.318566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984830af-cfe0-4de2-83b7-d46cf72cf935 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:31.320077 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984830af-cfe0-4de2-83b7-d46cf72cf935 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-984830af-cfe0-4de2-83b7-d46cf72cf935 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:06:31.332538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:31.333760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:31.339672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:06:31.339672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:06:31.339672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:06:31.339672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:06:31.339672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:06:31.339672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:06:31.339672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:06:31.339672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:31.339672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 612/1230] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:06:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:06:31.341775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb2d88-ace9-4828-a696-8e2b1fc406c4 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:31.459468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb2d88-ace9-4828-a696-8e2b1fc406c4 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:31.500287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce8abb-2da9-4a37-b6ef-9b403d800a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:31.500901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce8abb-2da9-4a37-b6ef-9b403d800a2c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:31.501177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce8abb-2da9-4a37-b6ef-9b403d800a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:31.501443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce8abb-2da9-4a37-b6ef-9b403d800a2c None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:31.502236 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce8abb-2da9-4a37-b6ef-9b403d800a2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:31.502628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce8abb-2da9-4a37-b6ef-9b403d800a2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:31.504248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce8abb-2da9-4a37-b6ef-9b403d800a2c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:31.519646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 619/1231] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:06:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:31.526303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984830af-cfe0-4de2-83b7-d46cf72cf935 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:31.601482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:31.646193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984830af-cfe0-4de2-83b7-d46cf72cf935 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:31.685868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f3898-c550-4507-a28a-14c0b2650d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:31.686565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f3898-c550-4507-a28a-14c0b2650d93 None admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:31.686842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f3898-c550-4507-a28a-14c0b2650d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:31.687118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f3898-c550-4507-a28a-14c0b2650d93 None admin] PATH_INFO: `/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/08f34295f19b4d6ea405f1e8c8190db5/roles/d3cef78b3402418ea77f177cc74fecb1` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:31.705132 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f3898-c550-4507-a28a-14c0b2650d93 None admin] RBAC: Authorizing `identity:create_grant(project_id=9188023bf51d4737b801e3fbd9af95b1, user_id=08f34295f19b4d6ea405f1e8c8190db5, role_id=d3cef78b3402418ea77f177cc74fecb1)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:31.706583 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f3898-c550-4507-a28a-14c0b2650d93 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:31.721596 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 613/1232] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:06:31 2022] PUT /identity/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/08f34295f19b4d6ea405f1e8c8190db5/roles/d3cef78b3402418ea77f177cc74fecb1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:06:33.827177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c735f-5f92-4b98-9cee-4cd95e7d2c9d None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:33.828055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c735f-5f92-4b98-9cee-4cd95e7d2c9d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:33.828055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c735f-5f92-4b98-9cee-4cd95e7d2c9d None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:33.829102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 620/1233] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:06:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:06:33.834679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2493519-2de8-4887-96c2-f1cf3b110bc9 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:33.835236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2493519-2de8-4887-96c2-f1cf3b110bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:33.835236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2493519-2de8-4887-96c2-f1cf3b110bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:33.856590 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-a2493519-2de8-4887-96c2-f1cf3b110bc9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:33.857796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a2493519-2de8-4887-96c2-f1cf3b110bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:33.935442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 614/1234] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:06:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:06:34.204368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c735f-5f92-4b98-9cee-4cd95e7d2c9d None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:34.233261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:34.329103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c735f-5f92-4b98-9cee-4cd95e7d2c9d None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:34.371222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8a7cf-285f-41b8-8b7e-5ca8ab221c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:34.371979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8a7cf-285f-41b8-8b7e-5ca8ab221c5e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:34.372319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8a7cf-285f-41b8-8b7e-5ca8ab221c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:34.372589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8a7cf-285f-41b8-8b7e-5ca8ab221c5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:34.385029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8a7cf-285f-41b8-8b7e-5ca8ab221c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-0ea8a7cf-285f-41b8-8b7e-5ca8ab221c5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:06:34.390505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:34.391749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:34.392734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 621/1235] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Aug 30 01:06:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:34.398229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2493519-2de8-4887-96c2-f1cf3b110bc9 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:34.456710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:34.520827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2493519-2de8-4887-96c2-f1cf3b110bc9 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:34.562427 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae334b-a294-4e55-93ca-585bc745c862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:34.563047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae334b-a294-4e55-93ca-585bc745c862 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:34.563322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae334b-a294-4e55-93ca-585bc745c862 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:34.563594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae334b-a294-4e55-93ca-585bc745c862 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:34.564442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae334b-a294-4e55-93ca-585bc745c862 None admin] RBAC: Adding query filter params (name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:34.564739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae334b-a294-4e55-93ca-585bc745c862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:34.566118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae334b-a294-4e55-93ca-585bc745c862 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:34.576399 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 615/1236] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Aug 30 01:06:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:06:34.582319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8a7cf-285f-41b8-8b7e-5ca8ab221c5e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:34.678437 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:34.696530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8a7cf-285f-41b8-8b7e-5ca8ab221c5e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:34.738301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba9766-e642-4cb3-af06-0e7f3d9396a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:34.738918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba9766-e642-4cb3-af06-0e7f3d9396a4 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:34.739222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba9766-e642-4cb3-af06-0e7f3d9396a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:34.739538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba9766-e642-4cb3-af06-0e7f3d9396a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:34.752473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba9766-e642-4cb3-af06-0e7f3d9396a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:34.754213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba9766-e642-4cb3-af06-0e7f3d9396a4 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-65ba9766-e642-4cb3-af06-0e7f3d9396a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:34.773567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:34.774708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:34.775729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:34.775729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:34.775729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:34.775729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 622/1237] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:34.782922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae334b-a294-4e55-93ca-585bc745c862 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:34.901917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae334b-a294-4e55-93ca-585bc745c862 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:34.914252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:34.943126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075eb4e-b917-4ab5-b5d1-0f83345e9436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:34.943773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075eb4e-b917-4ab5-b5d1-0f83345e9436 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:34.944082 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075eb4e-b917-4ab5-b5d1-0f83345e9436 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:34.944354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075eb4e-b917-4ab5-b5d1-0f83345e9436 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:34.945074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075eb4e-b917-4ab5-b5d1-0f83345e9436 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:34.945390 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075eb4e-b917-4ab5-b5d1-0f83345e9436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:34.946729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075eb4e-b917-4ab5-b5d1-0f83345e9436 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:34.964199 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 616/1238] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:34.969739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba9766-e642-4cb3-af06-0e7f3d9396a4 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:35.092201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba9766-e642-4cb3-af06-0e7f3d9396a4 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:35.134217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f82a3-9aed-4f8c-bd43-dcf452284794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:35.135061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f82a3-9aed-4f8c-bd43-dcf452284794 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:35.135340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f82a3-9aed-4f8c-bd43-dcf452284794 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:35.135626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f82a3-9aed-4f8c-bd43-dcf452284794 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:35.140161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:35.148247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f82a3-9aed-4f8c-bd43-dcf452284794 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:35.150577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f82a3-9aed-4f8c-bd43-dcf452284794 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-2d6f82a3-9aed-4f8c-bd43-dcf452284794 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:06:35.165565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:35.167393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:35.169323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:35.169323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:35.169323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:06:35.169323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:06:35.169323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:06:35.169323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:06:35.169323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:06:35.169323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:06:35.169323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:06:35.169323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:35.169323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 623/1239] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:06:35.174572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075eb4e-b917-4ab5-b5d1-0f83345e9436 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:35.306074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075eb4e-b917-4ab5-b5d1-0f83345e9436 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:35.350752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac5efb-c2aa-4764-bc20-2475f88dcf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:35.351422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac5efb-c2aa-4764-bc20-2475f88dcf2e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:35.351699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac5efb-c2aa-4764-bc20-2475f88dcf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:35.351971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac5efb-c2aa-4764-bc20-2475f88dcf2e None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:35.352815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac5efb-c2aa-4764-bc20-2475f88dcf2e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:35.353146 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac5efb-c2aa-4764-bc20-2475f88dcf2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:35.354799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac5efb-c2aa-4764-bc20-2475f88dcf2e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:35.378247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 617/1240] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Aug 30 01:06:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:06:35.384232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f82a3-9aed-4f8c-bd43-dcf452284794 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:35.498511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f82a3-9aed-4f8c-bd43-dcf452284794 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:35.540766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b6f60-d041-4544-bf83-6ee544060678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:35.541467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b6f60-d041-4544-bf83-6ee544060678 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:35.541775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b6f60-d041-4544-bf83-6ee544060678 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:35.542019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b6f60-d041-4544-bf83-6ee544060678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:35.615612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:35.623645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b6f60-d041-4544-bf83-6ee544060678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:35.625337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b6f60-d041-4544-bf83-6ee544060678 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:35.748694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 624/1241] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:06:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:06:35.754989 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac5efb-c2aa-4764-bc20-2475f88dcf2e None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:35.789973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:35.881698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac5efb-c2aa-4764-bc20-2475f88dcf2e None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:35.925015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71764e9b-86aa-4245-86f5-3213431c6aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:35.925840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71764e9b-86aa-4245-86f5-3213431c6aca None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:35.925939 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71764e9b-86aa-4245-86f5-3213431c6aca None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:35.926196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71764e9b-86aa-4245-86f5-3213431c6aca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:35.938619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71764e9b-86aa-4245-86f5-3213431c6aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:35.940301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71764e9b-86aa-4245-86f5-3213431c6aca None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-71764e9b-86aa-4245-86f5-3213431c6aca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:35.958857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:35.961118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:35.963126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:35.963126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:35.963126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:35.963126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 618/1242] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:35.969663 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b6f60-d041-4544-bf83-6ee544060678 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:36.092901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:36.099220 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b6f60-d041-4544-bf83-6ee544060678 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:36.140721 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b424ff7-81ca-4514-89c1-bf2afcaeb2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:36.141424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b424ff7-81ca-4514-89c1-bf2afcaeb2ee None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:36.141691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b424ff7-81ca-4514-89c1-bf2afcaeb2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:36.142010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b424ff7-81ca-4514-89c1-bf2afcaeb2ee None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:36.142752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b424ff7-81ca-4514-89c1-bf2afcaeb2ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:36.143096 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b424ff7-81ca-4514-89c1-bf2afcaeb2ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:36.144703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b424ff7-81ca-4514-89c1-bf2afcaeb2ee None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:36.158763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 625/1243] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:06:36.164381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71764e9b-86aa-4245-86f5-3213431c6aca None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:36.268840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:36.304703 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71764e9b-86aa-4245-86f5-3213431c6aca None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:36.346637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f157a95-624e-47cd-9b59-56a70bb3244f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:36.347538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f157a95-624e-47cd-9b59-56a70bb3244f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:36.347957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f157a95-624e-47cd-9b59-56a70bb3244f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:36.348397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f157a95-624e-47cd-9b59-56a70bb3244f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:36.359235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f157a95-624e-47cd-9b59-56a70bb3244f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:36.360940 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f157a95-624e-47cd-9b59-56a70bb3244f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-2f157a95-624e-47cd-9b59-56a70bb3244f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:06:36.378505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:36.379938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:36.381010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:06:36.381010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:06:36.381010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:06:36.381010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:06:36.381010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:06:36.381010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:06:36.381010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:06:36.381010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:36.381010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 619/1244] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:06:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:06:36.388882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b424ff7-81ca-4514-89c1-bf2afcaeb2ee None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:36.513709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b424ff7-81ca-4514-89c1-bf2afcaeb2ee None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:36.556427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b476fbe-a28e-4b00-aa32-78bd4bae8e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:36.557320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b476fbe-a28e-4b00-aa32-78bd4bae8e7a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:36.557716 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b476fbe-a28e-4b00-aa32-78bd4bae8e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:36.558079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b476fbe-a28e-4b00-aa32-78bd4bae8e7a None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:36.558905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b476fbe-a28e-4b00-aa32-78bd4bae8e7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:36.559725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b476fbe-a28e-4b00-aa32-78bd4bae8e7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:36.561103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b476fbe-a28e-4b00-aa32-78bd4bae8e7a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:36.579567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 626/1245] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:06:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:36.586532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f157a95-624e-47cd-9b59-56a70bb3244f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:36.710486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f157a95-624e-47cd-9b59-56a70bb3244f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:36.748801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a05fa-540c-4db0-a177-d8922d24be9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:36.749534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a05fa-540c-4db0-a177-d8922d24be9b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:36.749823 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a05fa-540c-4db0-a177-d8922d24be9b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:36.750096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a05fa-540c-4db0-a177-d8922d24be9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:36.750925 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a05fa-540c-4db0-a177-d8922d24be9b None admin] RBAC: Adding query filter params (role.id=d3cef78b3402418ea77f177cc74fecb1, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=08f34295f19b4d6ea405f1e8c8190db5) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:36.751249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a05fa-540c-4db0-a177-d8922d24be9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:36.752847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a05fa-540c-4db0-a177-d8922d24be9b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:36.759744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 620/1246] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:06:36 2022] GET /identity/v3/role_assignments?user.id=08f34295f19b4d6ea405f1e8c8190db5&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=d3cef78b3402418ea77f177cc74fecb1 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:38.821909 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e64e9-4563-4449-ad96-3841060d3f4c None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:38.822512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e64e9-4563-4449-ad96-3841060d3f4c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:38.822512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e64e9-4563-4449-ad96-3841060d3f4c None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:38.823962 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 627/1247] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:06:38.830175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c35639-662a-4d24-bbce-5228bad27e36 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:38.830837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c35639-662a-4d24-bbce-5228bad27e36 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:38.830932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c35639-662a-4d24-bbce-5228bad27e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:38.837946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:38.854009 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-39c35639-662a-4d24-bbce-5228bad27e36 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:38.855259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-39c35639-662a-4d24-bbce-5228bad27e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:38.938411 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 621/1248] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:06:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:06:39.236794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e64e9-4563-4449-ad96-3841060d3f4c None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:39.257128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:39.361160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e64e9-4563-4449-ad96-3841060d3f4c None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:39.402364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055da5e-7f0d-486a-a745-a1b64868b7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:39.403081 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055da5e-7f0d-486a-a745-a1b64868b7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:39.403384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055da5e-7f0d-486a-a745-a1b64868b7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:39.403675 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055da5e-7f0d-486a-a745-a1b64868b7d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:39.416316 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055da5e-7f0d-486a-a745-a1b64868b7d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-2055da5e-7f0d-486a-a745-a1b64868b7d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:06:39.422360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:39.423561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:39.424677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 628/1249] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:06:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:39.430247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c35639-662a-4d24-bbce-5228bad27e36 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:39.544289 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c35639-662a-4d24-bbce-5228bad27e36 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:39.574999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:39.580193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7f160-e9eb-4bc2-ba46-85ee295769a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:39.580885 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7f160-e9eb-4bc2-ba46-85ee295769a7 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:39.581157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7f160-e9eb-4bc2-ba46-85ee295769a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:39.581628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7f160-e9eb-4bc2-ba46-85ee295769a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:39.582476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7f160-e9eb-4bc2-ba46-85ee295769a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:39.582930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7f160-e9eb-4bc2-ba46-85ee295769a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:39.584628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7f160-e9eb-4bc2-ba46-85ee295769a7 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:39.595014 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 622/1250] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:06:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:06:39.600969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055da5e-7f0d-486a-a745-a1b64868b7d0 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:39.692549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:39.721564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055da5e-7f0d-486a-a745-a1b64868b7d0 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:39.763026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba047cb-157b-49bb-b518-65494cde57a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:39.763667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba047cb-157b-49bb-b518-65494cde57a6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:39.764009 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba047cb-157b-49bb-b518-65494cde57a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:39.764243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba047cb-157b-49bb-b518-65494cde57a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:39.775301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba047cb-157b-49bb-b518-65494cde57a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:39.776929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba047cb-157b-49bb-b518-65494cde57a6 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-cba047cb-157b-49bb-b518-65494cde57a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:39.794947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:39.801993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:39.803134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:39.803134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:39.803134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:39.803134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 629/1251] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:39.803400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7f160-e9eb-4bc2-ba46-85ee295769a7 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:39.925675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7f160-e9eb-4bc2-ba46-85ee295769a7 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:39.982299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce38ed-776e-4cdd-a3e7-cc1e0dd4b3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:39.982923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce38ed-776e-4cdd-a3e7-cc1e0dd4b3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:39.983197 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce38ed-776e-4cdd-a3e7-cc1e0dd4b3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:39.983466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce38ed-776e-4cdd-a3e7-cc1e0dd4b3a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:39.984162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce38ed-776e-4cdd-a3e7-cc1e0dd4b3a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:39.984441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce38ed-776e-4cdd-a3e7-cc1e0dd4b3a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:39.985806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce38ed-776e-4cdd-a3e7-cc1e0dd4b3a6 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:39.999744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 623/1252] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:06:40.005730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba047cb-157b-49bb-b518-65494cde57a6 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:40.126884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba047cb-157b-49bb-b518-65494cde57a6 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:40.159548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:40.167684 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eab53f-1312-4314-b903-f0cb236a712f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:40.168395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eab53f-1312-4314-b903-f0cb236a712f None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:40.168696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eab53f-1312-4314-b903-f0cb236a712f None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:40.169032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eab53f-1312-4314-b903-f0cb236a712f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:40.178089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eab53f-1312-4314-b903-f0cb236a712f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:40.179661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eab53f-1312-4314-b903-f0cb236a712f None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-b4eab53f-1312-4314-b903-f0cb236a712f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:06:40.193563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:40.194801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:40.196028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:40.196028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:40.196028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:06:40.196028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:06:40.196028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:06:40.196028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:06:40.196028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:06:40.196028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:06:40.196028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:06:40.196028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:40.196028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 630/1253] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:40.201609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce38ed-776e-4cdd-a3e7-cc1e0dd4b3a6 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:40.270621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:40.321433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce38ed-776e-4cdd-a3e7-cc1e0dd4b3a6 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:40.356290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61452cfe-6182-423e-89ad-8e46650f950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:40.356931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61452cfe-6182-423e-89ad-8e46650f950f None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:40.357211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61452cfe-6182-423e-89ad-8e46650f950f None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:40.357549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61452cfe-6182-423e-89ad-8e46650f950f None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:40.358209 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61452cfe-6182-423e-89ad-8e46650f950f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:40.358516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61452cfe-6182-423e-89ad-8e46650f950f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:40.359933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61452cfe-6182-423e-89ad-8e46650f950f None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:40.379296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 624/1254] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Aug 30 01:06:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:40.385299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eab53f-1312-4314-b903-f0cb236a712f None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:40.505928 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eab53f-1312-4314-b903-f0cb236a712f None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:40.544629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37905f-73b1-4a8b-8e92-93bb71190098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:40.545244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37905f-73b1-4a8b-8e92-93bb71190098 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:40.545615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37905f-73b1-4a8b-8e92-93bb71190098 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:40.545852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37905f-73b1-4a8b-8e92-93bb71190098 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:40.612199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:40.629764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37905f-73b1-4a8b-8e92-93bb71190098 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:40.631426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37905f-73b1-4a8b-8e92-93bb71190098 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:40.755787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 631/1255] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:06:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:06:40.764078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61452cfe-6182-423e-89ad-8e46650f950f None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:40.877544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61452cfe-6182-423e-89ad-8e46650f950f None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:40.895187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:40.921712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799d044-b470-4eb6-858f-4e8e5d4a72c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:40.922507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799d044-b470-4eb6-858f-4e8e5d4a72c0 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:40.922802 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799d044-b470-4eb6-858f-4e8e5d4a72c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:40.923068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799d044-b470-4eb6-858f-4e8e5d4a72c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:40.935848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799d044-b470-4eb6-858f-4e8e5d4a72c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:40.937350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799d044-b470-4eb6-858f-4e8e5d4a72c0 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-9799d044-b470-4eb6-858f-4e8e5d4a72c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:40.956055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:40.957521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:40.958604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:40.958604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:40.958604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:40.958604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 625/1256] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:40.963842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37905f-73b1-4a8b-8e92-93bb71190098 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:41.070559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:41.090076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37905f-73b1-4a8b-8e92-93bb71190098 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:41.131911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab309c-ce5e-4390-859a-cf36c6d1750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:41.132585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab309c-ce5e-4390-859a-cf36c6d1750b None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:41.132891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab309c-ce5e-4390-859a-cf36c6d1750b None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:41.133164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab309c-ce5e-4390-859a-cf36c6d1750b None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:41.133881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab309c-ce5e-4390-859a-cf36c6d1750b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:41.134221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab309c-ce5e-4390-859a-cf36c6d1750b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:41.135719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab309c-ce5e-4390-859a-cf36c6d1750b None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:41.151919 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 632/1257] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:41.159153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799d044-b470-4eb6-858f-4e8e5d4a72c0 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:41.284595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799d044-b470-4eb6-858f-4e8e5d4a72c0 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:41.321842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca2bd7-3ece-4f1b-8996-d9d24da7abef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:41.322475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca2bd7-3ece-4f1b-8996-d9d24da7abef None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:41.322768 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca2bd7-3ece-4f1b-8996-d9d24da7abef None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:41.323019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca2bd7-3ece-4f1b-8996-d9d24da7abef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:41.331330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca2bd7-3ece-4f1b-8996-d9d24da7abef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:41.332793 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca2bd7-3ece-4f1b-8996-d9d24da7abef None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-5eca2bd7-3ece-4f1b-8996-d9d24da7abef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:06:41.346107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:41.352054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:41.353214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:06:41.353214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:06:41.353214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:06:41.353214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:06:41.353214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:06:41.353214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:06:41.353214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:06:41.353214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:41.353214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 626/1258] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:06:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:41.354884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab309c-ce5e-4390-859a-cf36c6d1750b None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:41.468730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab309c-ce5e-4390-859a-cf36c6d1750b None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:41.509204 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98605994-476d-4492-a4d0-4da214f4c5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:41.509972 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98605994-476d-4492-a4d0-4da214f4c5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:41.510402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98605994-476d-4492-a4d0-4da214f4c5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:41.510717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98605994-476d-4492-a4d0-4da214f4c5f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:41.512085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98605994-476d-4492-a4d0-4da214f4c5f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:41.512482 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98605994-476d-4492-a4d0-4da214f4c5f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:41.514199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98605994-476d-4492-a4d0-4da214f4c5f6 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:41.528558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 633/1259] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:06:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:41.535627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca2bd7-3ece-4f1b-8996-d9d24da7abef None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:41.563901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:41.653781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca2bd7-3ece-4f1b-8996-d9d24da7abef None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:41.693103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b6d70-5356-4d21-a836-1954e044e35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:41.693719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b6d70-5356-4d21-a836-1954e044e35b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:41.694010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b6d70-5356-4d21-a836-1954e044e35b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:41.694352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b6d70-5356-4d21-a836-1954e044e35b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:41.695091 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b6d70-5356-4d21-a836-1954e044e35b None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=08f34295f19b4d6ea405f1e8c8190db5) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:41.695412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b6d70-5356-4d21-a836-1954e044e35b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:41.696943 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b6d70-5356-4d21-a836-1954e044e35b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:41.724178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 627/1260] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:06:41 2022] GET /identity/v3/role_assignments?user.id=08f34295f19b4d6ea405f1e8c8190db5&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:43.661956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e38b7-93dc-41e7-b84a-ea3f494a90e9 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:43.662496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e38b7-93dc-41e7-b84a-ea3f494a90e9 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:43.662496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e38b7-93dc-41e7-b84a-ea3f494a90e9 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:43.663837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 634/1261] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:06:43.669635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedb0a2-c66d-45d4-a4f9-6b32cba1063f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:43.670094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedb0a2-c66d-45d4-a4f9-6b32cba1063f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:43.670186 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedb0a2-c66d-45d4-a4f9-6b32cba1063f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:43.689643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8fedb0a2-c66d-45d4-a4f9-6b32cba1063f None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:43.690848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8fedb0a2-c66d-45d4-a4f9-6b32cba1063f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:43.759706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 628/1262] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:06:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:06:44.050350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e38b7-93dc-41e7-b84a-ea3f494a90e9 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:44.061484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:44.153498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e38b7-93dc-41e7-b84a-ea3f494a90e9 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:44.188646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00707c37-894b-4a91-9f1c-026901a8740e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:44.189272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00707c37-894b-4a91-9f1c-026901a8740e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:44.189552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00707c37-894b-4a91-9f1c-026901a8740e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:44.189824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00707c37-894b-4a91-9f1c-026901a8740e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:44.200265 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00707c37-894b-4a91-9f1c-026901a8740e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-00707c37-894b-4a91-9f1c-026901a8740e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:06:44.205075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:44.206169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:44.207233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 635/1263] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:06:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:44.212348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedb0a2-c66d-45d4-a4f9-6b32cba1063f None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:44.231511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:44.313942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedb0a2-c66d-45d4-a4f9-6b32cba1063f None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:44.349164 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03885ff-cba8-4d43-804b-44034061306a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:44.349774 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03885ff-cba8-4d43-804b-44034061306a None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:44.350050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03885ff-cba8-4d43-804b-44034061306a None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:44.350319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03885ff-cba8-4d43-804b-44034061306a None admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:44.350981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03885ff-cba8-4d43-804b-44034061306a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:44.351284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03885ff-cba8-4d43-804b-44034061306a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:44.352673 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03885ff-cba8-4d43-804b-44034061306a None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:44.361719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 629/1264] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:06:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:44.367666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00707c37-894b-4a91-9f1c-026901a8740e None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:44.434122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:44.468264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00707c37-894b-4a91-9f1c-026901a8740e None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:44.501739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab4579-371b-4daf-8f5e-735c7efb4537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:44.502357 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab4579-371b-4daf-8f5e-735c7efb4537 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:44.502653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab4579-371b-4daf-8f5e-735c7efb4537 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:44.502918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab4579-371b-4daf-8f5e-735c7efb4537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:44.513282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab4579-371b-4daf-8f5e-735c7efb4537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:44.514721 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab4579-371b-4daf-8f5e-735c7efb4537 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-d2ab4579-371b-4daf-8f5e-735c7efb4537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:44.530120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:44.531292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:44.532719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:44.532719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:44.532719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:44.532719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 636/1265] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:44.537386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03885ff-cba8-4d43-804b-44034061306a None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:44.623645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:44.644144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03885ff-cba8-4d43-804b-44034061306a None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:44.678783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbe5bb-b81f-4c14-b23d-4e30351cb8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:44.679523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbe5bb-b81f-4c14-b23d-4e30351cb8df None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:44.679646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbe5bb-b81f-4c14-b23d-4e30351cb8df None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:44.679914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbe5bb-b81f-4c14-b23d-4e30351cb8df None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:44.680582 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbe5bb-b81f-4c14-b23d-4e30351cb8df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:44.680883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbe5bb-b81f-4c14-b23d-4e30351cb8df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:44.682230 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbe5bb-b81f-4c14-b23d-4e30351cb8df None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:44.695804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 630/1266] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:06:44.702184 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab4579-371b-4daf-8f5e-735c7efb4537 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:44.803102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab4579-371b-4daf-8f5e-735c7efb4537 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:44.823023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:44.838376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eef6e-124a-404c-b84d-1ba49e7ba0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:44.839005 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eef6e-124a-404c-b84d-1ba49e7ba0ef None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:44.839499 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eef6e-124a-404c-b84d-1ba49e7ba0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:44.839730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eef6e-124a-404c-b84d-1ba49e7ba0ef None admin] PATH_INFO: `/v3/users/placement` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:44.847544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eef6e-124a-404c-b84d-1ba49e7ba0ef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:44.849022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eef6e-124a-404c-b84d-1ba49e7ba0ef None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-377eef6e-124a-404c-b84d-1ba49e7ba0ef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:06:44.862631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:44.863901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:44.864951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:44.864951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:44.864951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:06:44.864951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:06:44.864951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:06:44.864951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:06:44.864951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:06:44.864951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:06:44.864951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:06:44.864951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:44.864951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 637/1267] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:44.870152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbe5bb-b81f-4c14-b23d-4e30351cb8df None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:44.977726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbe5bb-b81f-4c14-b23d-4e30351cb8df None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:45.012209 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6073d2-5097-4a0b-8070-bb0097d00ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:45.012798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6073d2-5097-4a0b-8070-bb0097d00ccb None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:45.013071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6073d2-5097-4a0b-8070-bb0097d00ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:45.013337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6073d2-5097-4a0b-8070-bb0097d00ccb None admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:45.014057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6073d2-5097-4a0b-8070-bb0097d00ccb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:45.014353 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6073d2-5097-4a0b-8070-bb0097d00ccb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:45.015757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6073d2-5097-4a0b-8070-bb0097d00ccb None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:45.024308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:45.033983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 631/1268] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Aug 30 01:06:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:45.040049 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eef6e-124a-404c-b84d-1ba49e7ba0ef None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:45.161853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eef6e-124a-404c-b84d-1ba49e7ba0ef None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:45.219585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103053a-097d-4ba1-b65f-8fc1321a73f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:45.220698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103053a-097d-4ba1-b65f-8fc1321a73f2 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:45.221201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103053a-097d-4ba1-b65f-8fc1321a73f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:45.221585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103053a-097d-4ba1-b65f-8fc1321a73f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:45.240687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103053a-097d-4ba1-b65f-8fc1321a73f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:45.243423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103053a-097d-4ba1-b65f-8fc1321a73f2 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-3103053a-097d-4ba1-b65f-8fc1321a73f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:45.269659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:45.270861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:45.271908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:45.271908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:45.271908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:45.271908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 638/1269] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:45.278457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6073d2-5097-4a0b-8070-bb0097d00ccb None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:45.383148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6073d2-5097-4a0b-8070-bb0097d00ccb None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:45.418502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bee16-e1f7-4602-bc12-b9982373639c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:45.419110 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bee16-e1f7-4602-bc12-b9982373639c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:45.419389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bee16-e1f7-4602-bc12-b9982373639c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:45.419657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bee16-e1f7-4602-bc12-b9982373639c None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:45.420331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498bee16-e1f7-4602-bc12-b9982373639c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:45.420634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498bee16-e1f7-4602-bc12-b9982373639c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:45.421992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498bee16-e1f7-4602-bc12-b9982373639c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:45.433851 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 632/1270] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:06:45.440344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103053a-097d-4ba1-b65f-8fc1321a73f2 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:45.472881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:45.544170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103053a-097d-4ba1-b65f-8fc1321a73f2 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:45.577870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ad428-7da3-4d68-a5a2-154efb773a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:45.578514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ad428-7da3-4d68-a5a2-154efb773a8c None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:45.578787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ad428-7da3-4d68-a5a2-154efb773a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:45.579053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ad428-7da3-4d68-a5a2-154efb773a8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:45.587090 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ad428-7da3-4d68-a5a2-154efb773a8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:45.588939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ad428-7da3-4d68-a5a2-154efb773a8c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-3e7ad428-7da3-4d68-a5a2-154efb773a8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:06:45.602283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:45.603413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:45.605431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:06:45.605431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:06:45.605431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:06:45.605431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:06:45.605431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:06:45.605431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:06:45.605431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:06:45.605431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:45.605431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 639/1271] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:06:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:45.610891 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bee16-e1f7-4602-bc12-b9982373639c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:45.671603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:45.716116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bee16-e1f7-4602-bc12-b9982373639c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:45.750464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05858dff-70a6-455f-bc4d-5e366dc8a878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:45.751060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05858dff-70a6-455f-bc4d-5e366dc8a878 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:45.751340 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05858dff-70a6-455f-bc4d-5e366dc8a878 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:45.751610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05858dff-70a6-455f-bc4d-5e366dc8a878 None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:45.752330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05858dff-70a6-455f-bc4d-5e366dc8a878 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:45.752634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05858dff-70a6-455f-bc4d-5e366dc8a878 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:45.754058 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05858dff-70a6-455f-bc4d-5e366dc8a878 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:45.766177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 633/1272] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:06:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:45.772384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ad428-7da3-4d68-a5a2-154efb773a8c None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:45.862856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:45.872599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ad428-7da3-4d68-a5a2-154efb773a8c None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:45.907118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4e7f0-e6ac-4b53-9af3-d0dd1f0b978c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:45.907768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4e7f0-e6ac-4b53-9af3-d0dd1f0b978c None admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:45.908073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4e7f0-e6ac-4b53-9af3-d0dd1f0b978c None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:45.908343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4e7f0-e6ac-4b53-9af3-d0dd1f0b978c None admin] PATH_INFO: `/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/08f34295f19b4d6ea405f1e8c8190db5/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:45.910220 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4e7f0-e6ac-4b53-9af3-d0dd1f0b978c None admin] RBAC: Authorizing `identity:create_grant(project_id=9188023bf51d4737b801e3fbd9af95b1, user_id=08f34295f19b4d6ea405f1e8c8190db5, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:45.911657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4e7f0-e6ac-4b53-9af3-d0dd1f0b978c None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:45.924246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 640/1273] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Aug 30 01:06:45 2022] PUT /identity/v3/projects/9188023bf51d4737b801e3fbd9af95b1/users/08f34295f19b4d6ea405f1e8c8190db5/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:06:47.863160 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57e5ac-467b-4ba3-b6c3-4808f6bde980 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:47.863708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57e5ac-467b-4ba3-b6c3-4808f6bde980 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:47.863708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57e5ac-467b-4ba3-b6c3-4808f6bde980 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:47.865072 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 634/1274] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:06:47.870395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cff98-b032-446d-b72d-3f777275db7a None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:47.870947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cff98-b032-446d-b72d-3f777275db7a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:47.871150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cff98-b032-446d-b72d-3f777275db7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:47.889433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-385cff98-b032-446d-b72d-3f777275db7a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:47.890629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-385cff98-b032-446d-b72d-3f777275db7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:47.962578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 641/1275] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:06:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:06:48.242978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57e5ac-467b-4ba3-b6c3-4808f6bde980 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:48.344929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57e5ac-467b-4ba3-b6c3-4808f6bde980 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:48.362658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:48.379505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d9bd5-eb04-40bc-8860-1c763cd98288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:48.380221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d9bd5-eb04-40bc-8860-1c763cd98288 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:48.380450 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d9bd5-eb04-40bc-8860-1c763cd98288 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:48.380736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d9bd5-eb04-40bc-8860-1c763cd98288 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:48.391672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d9bd5-eb04-40bc-8860-1c763cd98288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-664d9bd5-eb04-40bc-8860-1c763cd98288 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:06:48.396529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:48.397773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:48.398799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 635/1276] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Aug 30 01:06:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:48.404452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cff98-b032-446d-b72d-3f777275db7a None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:48.490956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:48.511701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cff98-b032-446d-b72d-3f777275db7a None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:48.547367 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2c44d-73be-4e6b-809f-47fa7017485a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:48.547970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2c44d-73be-4e6b-809f-47fa7017485a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:48.548347 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2c44d-73be-4e6b-809f-47fa7017485a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:48.548530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2c44d-73be-4e6b-809f-47fa7017485a None admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:48.549199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2c44d-73be-4e6b-809f-47fa7017485a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:48.549495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2c44d-73be-4e6b-809f-47fa7017485a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:48.550862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2c44d-73be-4e6b-809f-47fa7017485a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:48.561563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 642/1277] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Aug 30 01:06:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:48.567394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d9bd5-eb04-40bc-8860-1c763cd98288 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:48.674837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d9bd5-eb04-40bc-8860-1c763cd98288 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:48.709225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf2aeb-30b8-459c-a75c-67de44ac638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:48.709873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf2aeb-30b8-459c-a75c-67de44ac638b None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:48.710147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf2aeb-30b8-459c-a75c-67de44ac638b None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:48.710417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf2aeb-30b8-459c-a75c-67de44ac638b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:48.723824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf2aeb-30b8-459c-a75c-67de44ac638b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:48.725329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf2aeb-30b8-459c-a75c-67de44ac638b None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-1ddf2aeb-30b8-459c-a75c-67de44ac638b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:48.740827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:48.742260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:48.743240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:48.743240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:48.743240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:48.743240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 636/1278] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:48.748396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2c44d-73be-4e6b-809f-47fa7017485a None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:48.854462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2c44d-73be-4e6b-809f-47fa7017485a None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:48.894218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77990b48-186b-4c03-b2a5-674288f20ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:48.895088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77990b48-186b-4c03-b2a5-674288f20ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:48.895493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77990b48-186b-4c03-b2a5-674288f20ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:48.895892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77990b48-186b-4c03-b2a5-674288f20ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:48.896886 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77990b48-186b-4c03-b2a5-674288f20ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:48.897342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77990b48-186b-4c03-b2a5-674288f20ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:48.899439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77990b48-186b-4c03-b2a5-674288f20ac2 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:48.915662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 643/1279] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:48.921585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf2aeb-30b8-459c-a75c-67de44ac638b None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:48.960236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:49.038042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf2aeb-30b8-459c-a75c-67de44ac638b None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:49.070272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e3aaa-3165-40eb-9e60-fc74b1c3df5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:49.070882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e3aaa-3165-40eb-9e60-fc74b1c3df5c None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:49.071155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e3aaa-3165-40eb-9e60-fc74b1c3df5c None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:49.071420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e3aaa-3165-40eb-9e60-fc74b1c3df5c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:49.079329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e3aaa-3165-40eb-9e60-fc74b1c3df5c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:49.080895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e3aaa-3165-40eb-9e60-fc74b1c3df5c None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-ec6e3aaa-3165-40eb-9e60-fc74b1c3df5c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:06:49.092468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:49.093591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:49.094568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:49.094568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:49.094568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:06:49.094568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:06:49.094568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:06:49.094568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:06:49.094568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:06:49.094568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:06:49.094568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:06:49.094568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:49.094568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 637/1280] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:49.099409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77990b48-186b-4c03-b2a5-674288f20ac2 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:49.103226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:49.202351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77990b48-186b-4c03-b2a5-674288f20ac2 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:49.248316 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1d55b-eb30-48df-8a22-e3a5609b413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:49.248932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1d55b-eb30-48df-8a22-e3a5609b413d None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:49.249211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1d55b-eb30-48df-8a22-e3a5609b413d None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:49.249489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1d55b-eb30-48df-8a22-e3a5609b413d None admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:49.250226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1d55b-eb30-48df-8a22-e3a5609b413d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:49.250533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1d55b-eb30-48df-8a22-e3a5609b413d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:49.252026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1d55b-eb30-48df-8a22-e3a5609b413d None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:49.271084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 644/1281] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Aug 30 01:06:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:49.277047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e3aaa-3165-40eb-9e60-fc74b1c3df5c None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:49.367265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:49.377574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e3aaa-3165-40eb-9e60-fc74b1c3df5c None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:49.411420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7449af-00a8-453c-87f6-19875f132c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:49.412059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7449af-00a8-453c-87f6-19875f132c31 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:49.412341 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7449af-00a8-453c-87f6-19875f132c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:49.412611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7449af-00a8-453c-87f6-19875f132c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:49.480281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7449af-00a8-453c-87f6-19875f132c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:49.481655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7449af-00a8-453c-87f6-19875f132c31 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:49.580638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:49.592904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 638/1282] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Aug 30 01:06:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:06:49.598921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1d55b-eb30-48df-8a22-e3a5609b413d None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:49.663841 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:49.706338 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1d55b-eb30-48df-8a22-e3a5609b413d None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:49.747322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d3d1f-9cc8-43ba-9c5e-d6f4bc1b6f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:49.747870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d3d1f-9cc8-43ba-9c5e-d6f4bc1b6f08 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:49.748168 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d3d1f-9cc8-43ba-9c5e-d6f4bc1b6f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:49.748434 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d3d1f-9cc8-43ba-9c5e-d6f4bc1b6f08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:49.760064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d3d1f-9cc8-43ba-9c5e-d6f4bc1b6f08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:49.762076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d3d1f-9cc8-43ba-9c5e-d6f4bc1b6f08 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-301d3d1f-9cc8-43ba-9c5e-d6f4bc1b6f08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:06:49.778978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:49.780210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:06:49.781849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:06:49.781849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:06:49.781849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:49.781849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 645/1283] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:49.786387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7449af-00a8-453c-87f6-19875f132c31 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:49.894530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7449af-00a8-453c-87f6-19875f132c31 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:49.939460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcca9bb-a4cb-46dd-9c8b-c94b36dfb2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:49.940302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcca9bb-a4cb-46dd-9c8b-c94b36dfb2cc None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:49.940397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcca9bb-a4cb-46dd-9c8b-c94b36dfb2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:49.940652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcca9bb-a4cb-46dd-9c8b-c94b36dfb2cc None admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:49.941311 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcca9bb-a4cb-46dd-9c8b-c94b36dfb2cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:49.941746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcca9bb-a4cb-46dd-9c8b-c94b36dfb2cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:49.943320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcca9bb-a4cb-46dd-9c8b-c94b36dfb2cc None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:49.957968 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 639/1284] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Aug 30 01:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:49.967312 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d3d1f-9cc8-43ba-9c5e-d6f4bc1b6f08 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:50.102584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d3d1f-9cc8-43ba-9c5e-d6f4bc1b6f08 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:50.117919 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:50.141230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256bafe-05f3-4de3-9d8b-fb2a7fbdc1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:50.141897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256bafe-05f3-4de3-9d8b-fb2a7fbdc1dd None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:50.142172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256bafe-05f3-4de3-9d8b-fb2a7fbdc1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:50.142443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256bafe-05f3-4de3-9d8b-fb2a7fbdc1dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:50.150736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256bafe-05f3-4de3-9d8b-fb2a7fbdc1dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:50.153123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256bafe-05f3-4de3-9d8b-fb2a7fbdc1dd None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application [None req-1256bafe-05f3-4de3-9d8b-fb2a7fbdc1dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:06:50.165938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:50.167100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:50.168268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:06:50.168268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:06:50.168268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:06:50.168268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:06:50.168268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:06:50.168268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:06:50.168268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:06:50.168268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: ERROR keystone.server.flask.application Aug 30 01:06:50.168268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 646/1285] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Aug 30 01:06:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:50.173714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcca9bb-a4cb-46dd-9c8b-c94b36dfb2cc None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:50.252437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:50.317805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcca9bb-a4cb-46dd-9c8b-c94b36dfb2cc None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:50.362851 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686ca1c-9d52-4305-b34b-1997924c30ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:50.363384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686ca1c-9d52-4305-b34b-1997924c30ce None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:50.363657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686ca1c-9d52-4305-b34b-1997924c30ce None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:50.363927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686ca1c-9d52-4305-b34b-1997924c30ce None admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:50.364656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686ca1c-9d52-4305-b34b-1997924c30ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:50.364971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686ca1c-9d52-4305-b34b-1997924c30ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:50.366847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686ca1c-9d52-4305-b34b-1997924c30ce None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:50.383122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 640/1286] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Aug 30 01:06:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:50.392309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256bafe-05f3-4de3-9d8b-fb2a7fbdc1dd None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:50.520417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256bafe-05f3-4de3-9d8b-fb2a7fbdc1dd None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:50.565143 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853e366-b470-4373-89fc-3fa227f1c64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:50.565751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853e366-b470-4373-89fc-3fa227f1c64a None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:50.566032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853e366-b470-4373-89fc-3fa227f1c64a None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:50.566308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853e366-b470-4373-89fc-3fa227f1c64a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:50.567043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853e366-b470-4373-89fc-3fa227f1c64a None admin] RBAC: Adding query filter params (role.id=1d3cf6d1581e45c8aff54a0afd66dd68, scope.project.id=9188023bf51d4737b801e3fbd9af95b1, user.id=08f34295f19b4d6ea405f1e8c8190db5) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:50.567354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853e366-b470-4373-89fc-3fa227f1c64a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:50.568858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853e366-b470-4373-89fc-3fa227f1c64a None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:50.575180 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 647/1287] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Aug 30 01:06:50 2022] GET /identity/v3/role_assignments?user.id=08f34295f19b4d6ea405f1e8c8190db5&scope.project.id=9188023bf51d4737b801e3fbd9af95b1&role.id=1d3cf6d1581e45c8aff54a0afd66dd68 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:52.541575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b396fa-9ceb-4eb3-855c-1976ebc5c2ce None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:52.542173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b396fa-9ceb-4eb3-855c-1976ebc5c2ce None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:52.542173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b396fa-9ceb-4eb3-855c-1976ebc5c2ce None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:52.543497 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 641/1288] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:06:52.548830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec3a51-c8d7-4732-bc1e-ac1d42ba47dc None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:52.549275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec3a51-c8d7-4732-bc1e-ac1d42ba47dc None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:52.549385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec3a51-c8d7-4732-bc1e-ac1d42ba47dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:52.567746 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-0aec3a51-c8d7-4732-bc1e-ac1d42ba47dc None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:52.568972 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0aec3a51-c8d7-4732-bc1e-ac1d42ba47dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:52.599098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:52.637234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 648/1289] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:06:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:06:52.904100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b396fa-9ceb-4eb3-855c-1976ebc5c2ce None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:53.033036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b396fa-9ceb-4eb3-855c-1976ebc5c2ce None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:53.044449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:53.068509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fcdd8-d20e-4596-8789-dadf0c752330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:53.069128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fcdd8-d20e-4596-8789-dadf0c752330 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:53.069405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fcdd8-d20e-4596-8789-dadf0c752330 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:53.069669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fcdd8-d20e-4596-8789-dadf0c752330 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:53.070245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fcdd8-d20e-4596-8789-dadf0c752330 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:53.071601 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fcdd8-d20e-4596-8789-dadf0c752330 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-161fcdd8-d20e-4596-8789-dadf0c752330 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:53.081949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:53.084270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:53.085194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 642/1290] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Aug 30 01:06:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:53.089738 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec3a51-c8d7-4732-bc1e-ac1d42ba47dc None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:53.192435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec3a51-c8d7-4732-bc1e-ac1d42ba47dc None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:53.229299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d4857-7e3c-4435-8b09-4d91199a5541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:53.229907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d4857-7e3c-4435-8b09-4d91199a5541 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:53.230182 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d4857-7e3c-4435-8b09-4d91199a5541 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:53.230450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d4857-7e3c-4435-8b09-4d91199a5541 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:53.231083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d4857-7e3c-4435-8b09-4d91199a5541 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:53.231388 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d4857-7e3c-4435-8b09-4d91199a5541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:53.232788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d4857-7e3c-4435-8b09-4d91199a5541 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:53.237786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 649/1291] 10.222.0.44 () {56 vars in 1116 bytes} [Tue Aug 30 01:06:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:06:53.243631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fcdd8-d20e-4596-8789-dadf0c752330 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:53.347529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fcdd8-d20e-4596-8789-dadf0c752330 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:53.382214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a8a01-c69b-4a0a-9a5e-687568e65639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:53.382817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a8a01-c69b-4a0a-9a5e-687568e65639 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:53.383094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a8a01-c69b-4a0a-9a5e-687568e65639 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:53.383365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a8a01-c69b-4a0a-9a5e-687568e65639 None admin] PATH_INFO: `/v3/services` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:53.384022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a8a01-c69b-4a0a-9a5e-687568e65639 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:53.384327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a8a01-c69b-4a0a-9a5e-687568e65639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:53.385858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a8a01-c69b-4a0a-9a5e-687568e65639 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:53.390718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 643/1292] 10.222.0.44 () {56 vars in 1116 bytes} [Tue Aug 30 01:06:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:55.295734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a03599-cef1-4250-81ff-5031bbacf742 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:55.296324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a03599-cef1-4250-81ff-5031bbacf742 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:55.296324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a03599-cef1-4250-81ff-5031bbacf742 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:55.297654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 650/1293] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:06:55.303477 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a24d6-184a-4dad-8c2e-3fff74c4aefb None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:55.303921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a24d6-184a-4dad-8c2e-3fff74c4aefb None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:55.304043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a24d6-184a-4dad-8c2e-3fff74c4aefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:55.321905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-5b4a24d6-184a-4dad-8c2e-3fff74c4aefb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:55.323171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5b4a24d6-184a-4dad-8c2e-3fff74c4aefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:55.352609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:55.390413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 644/1294] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:06:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:06:55.663578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a03599-cef1-4250-81ff-5031bbacf742 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:55.694249 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:55.764494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a03599-cef1-4250-81ff-5031bbacf742 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:55.799237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ae814-fd63-49c7-bcca-0f1c42993081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:55.799832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ae814-fd63-49c7-bcca-0f1c42993081 None admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:55.800133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ae814-fd63-49c7-bcca-0f1c42993081 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:55.800401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ae814-fd63-49c7-bcca-0f1c42993081 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:55.801057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ae814-fd63-49c7-bcca-0f1c42993081 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:55.802417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ae814-fd63-49c7-bcca-0f1c42993081 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:55.816521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 651/1295] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Aug 30 01:06:55 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:06:57.732739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890b938-4e8a-4306-94e4-8f382bb322de None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:57.733276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890b938-4e8a-4306-94e4-8f382bb322de None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:57.733276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890b938-4e8a-4306-94e4-8f382bb322de None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:57.734624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 645/1296] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:06:57.739844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b191d4-9a48-4163-8c83-1d34c7d4139d None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:57.740277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b191d4-9a48-4163-8c83-1d34c7d4139d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:57.740427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b191d4-9a48-4163-8c83-1d34c7d4139d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:57.758915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b7b191d4-9a48-4163-8c83-1d34c7d4139d None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:06:57.760740 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b7b191d4-9a48-4163-8c83-1d34c7d4139d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:06:57.834195 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 652/1297] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:06:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:06:58.116177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890b938-4e8a-4306-94e4-8f382bb322de None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:58.226567 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890b938-4e8a-4306-94e4-8f382bb322de None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:58.262724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db75d13-33ee-4923-b3f9-88297c45ee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:58.263355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db75d13-33ee-4923-b3f9-88297c45ee79 None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:58.263635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db75d13-33ee-4923-b3f9-88297c45ee79 None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:58.263904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db75d13-33ee-4923-b3f9-88297c45ee79 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:58.264523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db75d13-33ee-4923-b3f9-88297c45ee79 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:58.265877 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db75d13-33ee-4923-b3f9-88297c45ee79 None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-4db75d13-33ee-4923-b3f9-88297c45ee79 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:06:58.274796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:06:58.276815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:06:58.277985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 646/1298] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Aug 30 01:06:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:06:58.283498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b191d4-9a48-4163-8c83-1d34c7d4139d None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:58.302597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:58.388159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b191d4-9a48-4163-8c83-1d34c7d4139d None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:58.423958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540743a1-9c46-4a2e-8278-0bd72db8a8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:58.424605 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540743a1-9c46-4a2e-8278-0bd72db8a8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:58.424882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540743a1-9c46-4a2e-8278-0bd72db8a8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:58.425150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540743a1-9c46-4a2e-8278-0bd72db8a8b1 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:58.425781 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540743a1-9c46-4a2e-8278-0bd72db8a8b1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:58.426081 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540743a1-9c46-4a2e-8278-0bd72db8a8b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:58.427443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540743a1-9c46-4a2e-8278-0bd72db8a8b1 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:58.433239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 653/1299] 10.222.0.44 () {56 vars in 1116 bytes} [Tue Aug 30 01:06:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:58.439463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db75d13-33ee-4923-b3f9-88297c45ee79 None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:58.472482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:58.544629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db75d13-33ee-4923-b3f9-88297c45ee79 None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:58.578576 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdf56b-1dfd-487e-9f57-bbda0216d58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:58.579170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdf56b-1dfd-487e-9f57-bbda0216d58e None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:58.579446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdf56b-1dfd-487e-9f57-bbda0216d58e None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:58.579717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdf56b-1dfd-487e-9f57-bbda0216d58e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:58.580439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdf56b-1dfd-487e-9f57-bbda0216d58e None admin] RBAC: Adding query filter params (interface=public, service_id=de5f5e4e56f9401aa26cd624ff026ac9, region_id=RegionOne) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:06:58.580744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdf56b-1dfd-487e-9f57-bbda0216d58e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:58.582098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdf56b-1dfd-487e-9f57-bbda0216d58e None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:58.586974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 647/1300] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Aug 30 01:06:58 2022] GET /identity/v3/endpoints?service_id=de5f5e4e56f9401aa26cd624ff026ac9&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:06:58.593308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540743a1-9c46-4a2e-8278-0bd72db8a8b1 None admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:06:58.674361 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:06:58.693492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540743a1-9c46-4a2e-8278-0bd72db8a8b1 None admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:06:58.730358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3dc4a9-6733-4018-b235-72cb0c6261f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:06:58.730975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dc4a9-6733-4018-b235-72cb0c6261f0 None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:06:58.731256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dc4a9-6733-4018-b235-72cb0c6261f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:06:58.731528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dc4a9-6733-4018-b235-72cb0c6261f0 None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:06:58.732189 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3dc4a9-6733-4018-b235-72cb0c6261f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:06:58.733582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3dc4a9-6733-4018-b235-72cb0c6261f0 None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:06:58.740491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 654/1301] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Aug 30 01:06:58 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:00.680366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270fddd-c099-4e17-9c95-0af4a14ecece None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.680902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270fddd-c099-4e17-9c95-0af4a14ecece None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.680902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270fddd-c099-4e17-9c95-0af4a14ecece None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.682283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 648/1302] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:07:00.687752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443eb31d-188d-4333-bc54-136f9ab93546 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:00.688253 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443eb31d-188d-4333-bc54-136f9ab93546 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:00.688342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443eb31d-188d-4333-bc54-136f9ab93546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:00.706866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-443eb31d-188d-4333-bc54-136f9ab93546 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:00.708080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-443eb31d-188d-4333-bc54-136f9ab93546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:00.776165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 655/1303] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:07:00 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:07:01.050704 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270fddd-c099-4e17-9c95-0af4a14ecece None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:01.157844 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:01.166599 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270fddd-c099-4e17-9c95-0af4a14ecece None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:01.204398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fea06-beec-4aa8-b531-b5f125d898ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:01.205036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fea06-beec-4aa8-b531-b5f125d898ba None admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:01.205315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fea06-beec-4aa8-b531-b5f125d898ba None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:01.205587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fea06-beec-4aa8-b531-b5f125d898ba None admin] PATH_INFO: `/v3/services/placement` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:01.206172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fea06-beec-4aa8-b531-b5f125d898ba None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:01.207531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fea06-beec-4aa8-b531-b5f125d898ba None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application [None req-e08fea06-beec-4aa8-b531-b5f125d898ba None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:07:01.216297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:07:01.217461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application with Lock( Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: ERROR keystone.server.flask.application Aug 30 01:07:01.218618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 649/1304] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Aug 30 01:07:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:07:01.223886 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443eb31d-188d-4333-bc54-136f9ab93546 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:01.305759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:01.332230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443eb31d-188d-4333-bc54-136f9ab93546 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:01.367235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdaa3fb-886c-4a57-ac6a-19acc692bc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:01.367843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdaa3fb-886c-4a57-ac6a-19acc692bc1e None admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:01.368152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdaa3fb-886c-4a57-ac6a-19acc692bc1e None admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:01.369301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdaa3fb-886c-4a57-ac6a-19acc692bc1e None admin] PATH_INFO: `/v3/services` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:01.370005 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdaa3fb-886c-4a57-ac6a-19acc692bc1e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:07:01.370304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdaa3fb-886c-4a57-ac6a-19acc692bc1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:01.371683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdaa3fb-886c-4a57-ac6a-19acc692bc1e None admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:01.377250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 656/1305] 10.222.0.44 () {56 vars in 1116 bytes} [Tue Aug 30 01:07:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:07:01.383060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fea06-beec-4aa8-b531-b5f125d898ba None admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:01.487069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fea06-beec-4aa8-b531-b5f125d898ba None admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:01.522541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3666c-309d-4d43-a927-d9b9744e05aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:01.523143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3666c-309d-4d43-a927-d9b9744e05aa None admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:01.523420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3666c-309d-4d43-a927-d9b9744e05aa None admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:01.523819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3666c-309d-4d43-a927-d9b9744e05aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:01.524730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3666c-309d-4d43-a927-d9b9744e05aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:01.526788 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3666c-309d-4d43-a927-d9b9744e05aa None admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:01.546938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 650/1306] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:11.374120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=30 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:11.542863 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:07.654960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855ca1c-03a3-45a2-9e21-16887b1ec393 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:07.656066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855ca1c-03a3-45a2-9e21-16887b1ec393 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:07.656489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855ca1c-03a3-45a2-9e21-16887b1ec393 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:07.658909 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 657/1307] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:07.665597 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549531d-c9ce-4fe1-9ac8-fe9b475a631d None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:07.666498 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549531d-c9ce-4fe1-9ac8-fe9b475a631d None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:07.667078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549531d-c9ce-4fe1-9ac8-fe9b475a631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:07.826811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d549531d-c9ce-4fe1-9ac8-fe9b475a631d None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:07.828362 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d549531d-c9ce-4fe1-9ac8-fe9b475a631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:08.007314 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 651/1308] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:08.389573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac599e-f3e8-4a3f-af9d-ff07124b7384 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:08.390045 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac599e-f3e8-4a3f-af9d-ff07124b7384 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:08.390469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac599e-f3e8-4a3f-af9d-ff07124b7384 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:08.392857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 658/1309] 10.222.0.44 () {54 vars in 965 bytes} [Tue Aug 30 01:08:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:08.400729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed70cb3-a620-4d44-88e0-140b9a0e3a5f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:08.401165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed70cb3-a620-4d44-88e0-140b9a0e3a5f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:08.401548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed70cb3-a620-4d44-88e0-140b9a0e3a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:08.544723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-3ed70cb3-a620-4d44-88e0-140b9a0e3a5f None None] MFA Rules not processed for user `86cea2f9238b42e0a7cb26bac494a8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:08.545883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3ed70cb3-a620-4d44-88e0-140b9a0e3a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:08.571769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:08.639088 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 652/1310] 10.222.0.44 () {58 vars in 1062 bytes} [Tue Aug 30 01:08:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:08.649297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac599e-f3e8-4a3f-af9d-ff07124b7384 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:08.651135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-76ac599e-f3e8-4a3f-af9d-ff07124b7384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:08.772288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac599e-f3e8-4a3f-af9d-ff07124b7384 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:08.853887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5de30f-04d6-44b6-9921-3176b050c800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:08.854414 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5de30f-04d6-44b6-9921-3176b050c800 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:08.854690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5de30f-04d6-44b6-9921-3176b050c800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:08.854965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5de30f-04d6-44b6-9921-3176b050c800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:08.856648 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-fa5de30f-04d6-44b6-9921-3176b050c800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:08.923540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:08.995027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5de30f-04d6-44b6-9921-3176b050c800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:08.996813 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5de30f-04d6-44b6-9921-3176b050c800 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:09.048625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 659/1311] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:11.318896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae262aa-ab85-4068-b5a8-8d2b3f320170 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:11.319486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae262aa-ab85-4068-b5a8-8d2b3f320170 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:11.319486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae262aa-ab85-4068-b5a8-8d2b3f320170 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:11.321051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 653/1312] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:11.326569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8256e02-05a2-41f6-945d-ea869e72d966 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:11.327081 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8256e02-05a2-41f6-945d-ea869e72d966 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:11.327201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8256e02-05a2-41f6-945d-ea869e72d966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:11.400781 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-a8256e02-05a2-41f6-945d-ea869e72d966 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:11.402071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a8256e02-05a2-41f6-945d-ea869e72d966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:11.500215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 660/1313] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:11.784604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae262aa-ab85-4068-b5a8-8d2b3f320170 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:11.786225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0ae262aa-ab85-4068-b5a8-8d2b3f320170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:11.897322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae262aa-ab85-4068-b5a8-8d2b3f320170 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:11.971819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:11.974609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d64ee-b5a9-4c60-8507-45a661c81bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:11.975433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d64ee-b5a9-4c60-8507-45a661c81bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:11.975939 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d64ee-b5a9-4c60-8507-45a661c81bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:11.976457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d64ee-b5a9-4c60-8507-45a661c81bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:11.978086 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-bf4d64ee-b5a9-4c60-8507-45a661c81bf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:12.089657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d64ee-b5a9-4c60-8507-45a661c81bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:12.091103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d64ee-b5a9-4c60-8507-45a661c81bf7 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:12.110952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 654/1314] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:14.349912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0dc5d-9768-429f-b18d-752f4a3211ba None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:14.350459 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0dc5d-9768-429f-b18d-752f4a3211ba None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:14.350722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0dc5d-9768-429f-b18d-752f4a3211ba None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:14.352130 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 661/1315] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:14.357510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61638d7-b9dc-48d5-88d9-4e27e7db95a7 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:14.357943 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61638d7-b9dc-48d5-88d9-4e27e7db95a7 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:14.358051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61638d7-b9dc-48d5-88d9-4e27e7db95a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:14.425799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c61638d7-b9dc-48d5-88d9-4e27e7db95a7 None None] MFA Rules not processed for user `1370863b9efe4da78c3d0201098c91b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:14.427020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c61638d7-b9dc-48d5-88d9-4e27e7db95a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:14.479251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:14.526777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 655/1316] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:14 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:14.811841 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0dc5d-9768-429f-b18d-752f4a3211ba None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:14.814140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-99b0dc5d-9768-429f-b18d-752f4a3211ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:14.824383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:14.921228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0dc5d-9768-429f-b18d-752f4a3211ba None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:14.996140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d542d-f968-43bf-aff7-248c26aabe39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:14.996658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d542d-f968-43bf-aff7-248c26aabe39 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:14.996937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d542d-f968-43bf-aff7-248c26aabe39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:14.997205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d542d-f968-43bf-aff7-248c26aabe39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:14.998823 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-146d542d-f968-43bf-aff7-248c26aabe39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:15.061865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:15.117880 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d542d-f968-43bf-aff7-248c26aabe39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:15.119329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d542d-f968-43bf-aff7-248c26aabe39 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:15.137941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 662/1317] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:14 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:17.644141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03c767-1d4e-4728-9c19-b7b8748761d9 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:17.644663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03c767-1d4e-4728-9c19-b7b8748761d9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:17.644663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03c767-1d4e-4728-9c19-b7b8748761d9 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:17.646006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 656/1318] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:17.651348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae8764-976c-4b02-b557-c89c9bbb2003 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:17.651773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae8764-976c-4b02-b557-c89c9bbb2003 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:17.652025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae8764-976c-4b02-b557-c89c9bbb2003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:17.719703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7cae8764-976c-4b02-b557-c89c9bbb2003 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:17.720930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7cae8764-976c-4b02-b557-c89c9bbb2003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:17.818037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 663/1319] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:18.109902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e7438-8e1c-4d16-af01-fbc0dc5fd9ee None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.110187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e7438-8e1c-4d16-af01-fbc0dc5fd9ee None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.110449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e7438-8e1c-4d16-af01-fbc0dc5fd9ee None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.111818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 657/1320] 10.222.0.44 () {54 vars in 965 bytes} [Tue Aug 30 01:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:18.117906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecb944-aae1-4a4f-8650-16d6b50c1a3d None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.118191 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecb944-aae1-4a4f-8650-16d6b50c1a3d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.118462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecb944-aae1-4a4f-8650-16d6b50c1a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.141176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:18.198540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-80ecb944-aae1-4a4f-8650-16d6b50c1a3d None None] MFA Rules not processed for user `86cea2f9238b42e0a7cb26bac494a8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:18.199750 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-80ecb944-aae1-4a4f-8650-16d6b50c1a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:18.277217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 664/1321] 10.222.0.44 () {58 vars in 1062 bytes} [Tue Aug 30 01:08:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:18.286696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e7438-8e1c-4d16-af01-fbc0dc5fd9ee None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:18.288262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-939e7438-8e1c-4d16-af01-fbc0dc5fd9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:18.394148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e7438-8e1c-4d16-af01-fbc0dc5fd9ee None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:18.470942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24b513-a337-41f4-a810-182b2c99b359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:18.471546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24b513-a337-41f4-a810-182b2c99b359 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:18.471825 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24b513-a337-41f4-a810-182b2c99b359 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:18.472123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24b513-a337-41f4-a810-182b2c99b359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:18.473788 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-af24b513-a337-41f4-a810-182b2c99b359 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:18.481274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:18.589805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24b513-a337-41f4-a810-182b2c99b359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:18.591461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24b513-a337-41f4-a810-182b2c99b359 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:18.614110 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 658/1322] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:21.773059 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05132beb-5eab-47c1-acf8-6bb8538f0b67 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:21.773589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05132beb-5eab-47c1-acf8-6bb8538f0b67 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:21.773589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05132beb-5eab-47c1-acf8-6bb8538f0b67 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:21.774978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 665/1323] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:21.781981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aec41c-ac9b-4a89-8154-6169c5b74933 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:21.782468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aec41c-ac9b-4a89-8154-6169c5b74933 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:21.782562 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aec41c-ac9b-4a89-8154-6169c5b74933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:21.852102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-34aec41c-ac9b-4a89-8154-6169c5b74933 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:21.853299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-34aec41c-ac9b-4a89-8154-6169c5b74933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:21.916508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:21.960014 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 659/1324] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:22.241655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05132beb-5eab-47c1-acf8-6bb8538f0b67 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:22.243225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-05132beb-5eab-47c1-acf8-6bb8538f0b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:22.353791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05132beb-5eab-47c1-acf8-6bb8538f0b67 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:22.366593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:22.429825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc0b45-4070-42d0-bbe8-b399a0975bf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:22.430435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc0b45-4070-42d0-bbe8-b399a0975bf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:22.430705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc0b45-4070-42d0-bbe8-b399a0975bf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:22.430968 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc0b45-4070-42d0-bbe8-b399a0975bf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:22.432576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-15dc0b45-4070-42d0-bbe8-b399a0975bf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:22.546496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc0b45-4070-42d0-bbe8-b399a0975bf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:22.547863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc0b45-4070-42d0-bbe8-b399a0975bf0 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:22.568165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 666/1325] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:24.566944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decad9c9-cc93-464b-ba0e-405465192038 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:24.567602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decad9c9-cc93-464b-ba0e-405465192038 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:24.567602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decad9c9-cc93-464b-ba0e-405465192038 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:24.568777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 660/1326] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:24.574181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e8d9c-51e2-467d-b626-a481b3fbbf17 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:24.574783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e8d9c-51e2-467d-b626-a481b3fbbf17 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:24.574783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e8d9c-51e2-467d-b626-a481b3fbbf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:24.625116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:24.645142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-574e8d9c-51e2-467d-b626-a481b3fbbf17 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:24.646371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-574e8d9c-51e2-467d-b626-a481b3fbbf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:24.751933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 667/1327] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:25.041303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decad9c9-cc93-464b-ba0e-405465192038 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:25.042858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-decad9c9-cc93-464b-ba0e-405465192038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:25.154661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decad9c9-cc93-464b-ba0e-405465192038 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:25.240036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0363301-148b-4c77-bbb5-b91bc8a839db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:25.240692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0363301-148b-4c77-bbb5-b91bc8a839db service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:25.241001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0363301-148b-4c77-bbb5-b91bc8a839db service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:25.241270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0363301-148b-4c77-bbb5-b91bc8a839db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:25.242933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d0363301-148b-4c77-bbb5-b91bc8a839db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:25.261788 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:25.373007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0363301-148b-4c77-bbb5-b91bc8a839db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:25.375140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0363301-148b-4c77-bbb5-b91bc8a839db service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:25.396805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 661/1328] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:27.521812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aae32d-2ae1-4afc-a309-00eb8b732aa5 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:27.522648 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aae32d-2ae1-4afc-a309-00eb8b732aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:27.522648 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aae32d-2ae1-4afc-a309-00eb8b732aa5 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:27.524129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 668/1329] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:27.531175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50999c4-37bd-4e88-9462-d2b39ae0d994 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:27.531726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50999c4-37bd-4e88-9462-d2b39ae0d994 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:27.531726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50999c4-37bd-4e88-9462-d2b39ae0d994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:27.623238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c50999c4-37bd-4e88-9462-d2b39ae0d994 None None] MFA Rules not processed for user `1370863b9efe4da78c3d0201098c91b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:27.624588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c50999c4-37bd-4e88-9462-d2b39ae0d994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:27.699665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:27.741275 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 662/1330] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:28.041259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aae32d-2ae1-4afc-a309-00eb8b732aa5 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:28.042833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-27aae32d-2ae1-4afc-a309-00eb8b732aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:28.153012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aae32d-2ae1-4afc-a309-00eb8b732aa5 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:28.174285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:28.230992 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995092fb-99dc-4876-8a0d-a6eed6a36acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:28.231515 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995092fb-99dc-4876-8a0d-a6eed6a36acb service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:28.231792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995092fb-99dc-4876-8a0d-a6eed6a36acb service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:28.232102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995092fb-99dc-4876-8a0d-a6eed6a36acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:28.233757 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-995092fb-99dc-4876-8a0d-a6eed6a36acb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:28.348472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995092fb-99dc-4876-8a0d-a6eed6a36acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:28.349922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995092fb-99dc-4876-8a0d-a6eed6a36acb service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:28.368553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 669/1331] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:30.585246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68568045-d916-4d1d-a837-8466dd803ae1 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.585812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68568045-d916-4d1d-a837-8466dd803ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.585812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68568045-d916-4d1d-a837-8466dd803ae1 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.587101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 663/1332] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:30.593301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f85d3-fffd-4f7f-9ca9-937b82650251 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:30.593779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f85d3-fffd-4f7f-9ca9-937b82650251 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:30.593902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f85d3-fffd-4f7f-9ca9-937b82650251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:30.657652 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:30.660440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-a81f85d3-fffd-4f7f-9ca9-937b82650251 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:30.662236 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a81f85d3-fffd-4f7f-9ca9-937b82650251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:30.759364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 670/1333] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:31.039259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68568045-d916-4d1d-a837-8466dd803ae1 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:31.040828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-68568045-d916-4d1d-a837-8466dd803ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:31.145494 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68568045-d916-4d1d-a837-8466dd803ae1 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:31.223154 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e604a-4b67-4523-9397-d2940339a9fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:31.223751 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e604a-4b67-4523-9397-d2940339a9fe service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:31.224043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e604a-4b67-4523-9397-d2940339a9fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:31.224312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e604a-4b67-4523-9397-d2940339a9fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:31.225897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-124e604a-4b67-4523-9397-d2940339a9fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:31.253503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:31.335391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e604a-4b67-4523-9397-d2940339a9fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:31.336772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e604a-4b67-4523-9397-d2940339a9fe service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:31.354882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 664/1334] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:33.314456 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50643e-fa17-475a-8f79-d7dd6193c666 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:33.315043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50643e-fa17-475a-8f79-d7dd6193c666 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:33.315043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50643e-fa17-475a-8f79-d7dd6193c666 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:33.316399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 671/1335] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:33.321449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb1864-3c1e-49e9-aabd-81a0fa08bd6c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:33.321842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb1864-3c1e-49e9-aabd-81a0fa08bd6c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:33.321993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb1864-3c1e-49e9-aabd-81a0fa08bd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:33.386571 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c6bb1864-3c1e-49e9-aabd-81a0fa08bd6c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:33.387772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c6bb1864-3c1e-49e9-aabd-81a0fa08bd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:33.466030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:33.489772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 665/1336] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:33.775341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50643e-fa17-475a-8f79-d7dd6193c666 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:33.776918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2f50643e-fa17-475a-8f79-d7dd6193c666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:33.882292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50643e-fa17-475a-8f79-d7dd6193c666 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:33.910811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:33.957404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb0779-c592-42ac-9551-0c842d997fe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:33.958137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb0779-c592-42ac-9551-0c842d997fe1 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:33.958419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb0779-c592-42ac-9551-0c842d997fe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:33.958690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb0779-c592-42ac-9551-0c842d997fe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:33.960393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-09fb0779-c592-42ac-9551-0c842d997fe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:34.071607 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb0779-c592-42ac-9551-0c842d997fe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:34.073102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb0779-c592-42ac-9551-0c842d997fe1 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:34.090677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 672/1337] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:36.444718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823fae6-2802-477d-b9ab-0880ab29936a None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:36.445258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823fae6-2802-477d-b9ab-0880ab29936a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:36.445258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823fae6-2802-477d-b9ab-0880ab29936a None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:36.446602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 666/1338] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:36.452832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33715103-5f63-48e8-acfb-911dba3429e9 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:36.453271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33715103-5f63-48e8-acfb-911dba3429e9 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:36.453387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33715103-5f63-48e8-acfb-911dba3429e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:36.529775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-33715103-5f63-48e8-acfb-911dba3429e9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:36.531005 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-33715103-5f63-48e8-acfb-911dba3429e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:36.539529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:36.643075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 673/1339] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:36.925217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823fae6-2802-477d-b9ab-0880ab29936a None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:36.926765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0823fae6-2802-477d-b9ab-0880ab29936a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:37.040319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823fae6-2802-477d-b9ab-0880ab29936a None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:37.114308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2409a80-7779-43e3-9e95-5d0569aa0039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:37.114917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2409a80-7779-43e3-9e95-5d0569aa0039 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:37.115191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2409a80-7779-43e3-9e95-5d0569aa0039 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:37.115460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2409a80-7779-43e3-9e95-5d0569aa0039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:37.117102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e2409a80-7779-43e3-9e95-5d0569aa0039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:37.157813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:37.237459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2409a80-7779-43e3-9e95-5d0569aa0039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:37.238869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2409a80-7779-43e3-9e95-5d0569aa0039 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:37.257481 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 667/1340] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:41.352792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43650ff-d940-4943-b96c-0ae836f360d0 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:41.353353 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43650ff-d940-4943-b96c-0ae836f360d0 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:41.353473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43650ff-d940-4943-b96c-0ae836f360d0 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:41.354898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 674/1341] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:41.360565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f99c56-c8d9-4056-bd61-76f6cd449a1f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:41.361018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f99c56-c8d9-4056-bd61-76f6cd449a1f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:41.361116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f99c56-c8d9-4056-bd61-76f6cd449a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:41.432392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-e2f99c56-c8d9-4056-bd61-76f6cd449a1f None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:41.433643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e2f99c56-c8d9-4056-bd61-76f6cd449a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:41.531756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:41.543845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 668/1342] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:41.832036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43650ff-d940-4943-b96c-0ae836f360d0 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:41.833594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e43650ff-d940-4943-b96c-0ae836f360d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:41.953721 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43650ff-d940-4943-b96c-0ae836f360d0 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:41.994948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:42.038115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fa212-3314-450b-96e1-732c1d091cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:42.038724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fa212-3314-450b-96e1-732c1d091cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:42.039020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fa212-3314-450b-96e1-732c1d091cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:42.039283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fa212-3314-450b-96e1-732c1d091cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:42.040957 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8b7fa212-3314-450b-96e1-732c1d091cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:42.159843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fa212-3314-450b-96e1-732c1d091cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:42.161318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fa212-3314-450b-96e1-732c1d091cd1 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:42.180195 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 675/1343] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:44.806356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5672d6-516f-4c97-b8e9-84981cbc7912 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:44.806948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5672d6-516f-4c97-b8e9-84981cbc7912 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:44.806948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5672d6-516f-4c97-b8e9-84981cbc7912 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:44.808343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 669/1344] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:44.814273 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f781993-9d61-4b39-a911-83adb7ed8e15 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:44.814824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f781993-9d61-4b39-a911-83adb7ed8e15 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:44.814824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f781993-9d61-4b39-a911-83adb7ed8e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:44.887907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7f781993-9d61-4b39-a911-83adb7ed8e15 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:44.889300 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7f781993-9d61-4b39-a911-83adb7ed8e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:44.908981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:44.990440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 676/1345] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:45.276465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5672d6-516f-4c97-b8e9-84981cbc7912 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:45.278391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8e5672d6-516f-4c97-b8e9-84981cbc7912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:45.389165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5672d6-516f-4c97-b8e9-84981cbc7912 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:45.478757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963fe17-bb95-4f86-a540-b1166d684d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:45.479365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963fe17-bb95-4f86-a540-b1166d684d80 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:45.479642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963fe17-bb95-4f86-a540-b1166d684d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:45.479906 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963fe17-bb95-4f86-a540-b1166d684d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:45.482179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1963fe17-bb95-4f86-a540-b1166d684d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:45.535613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:45.616378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963fe17-bb95-4f86-a540-b1166d684d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:45.617856 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963fe17-bb95-4f86-a540-b1166d684d80 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:45.638429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 670/1346] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:49.871219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c835b7f-f2d3-438f-8fa6-69502ca2a4d8 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:49.871794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c835b7f-f2d3-438f-8fa6-69502ca2a4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:49.871794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c835b7f-f2d3-438f-8fa6-69502ca2a4d8 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:49.877700 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 677/1347] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:08:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:49.879470 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8fe59-7aee-4ea2-9343-d36707699782 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:49.879939 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8fe59-7aee-4ea2-9343-d36707699782 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:49.880047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8fe59-7aee-4ea2-9343-d36707699782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:49.957048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-81e8fe59-7aee-4ea2-9343-d36707699782 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:49.958344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-81e8fe59-7aee-4ea2-9343-d36707699782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:50.064402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:50.067194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 671/1348] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:08:50.351969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c835b7f-f2d3-438f-8fa6-69502ca2a4d8 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:50.353624 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6c835b7f-f2d3-438f-8fa6-69502ca2a4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:50.463597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c835b7f-f2d3-438f-8fa6-69502ca2a4d8 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:50.510887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:50.542047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba208c-42d2-4b4d-bab0-35f6ea12b8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:50.543334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba208c-42d2-4b4d-bab0-35f6ea12b8ce service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:50.543616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba208c-42d2-4b4d-bab0-35f6ea12b8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:50.543887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba208c-42d2-4b4d-bab0-35f6ea12b8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:50.545582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3fba208c-42d2-4b4d-bab0-35f6ea12b8ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:50.655970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba208c-42d2-4b4d-bab0-35f6ea12b8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:50.657500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba208c-42d2-4b4d-bab0-35f6ea12b8ce service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:50.676381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 678/1349] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:55.916125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df270c81-61ac-4494-9504-1773dfe5367c None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.916125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df270c81-61ac-4494-9504-1773dfe5367c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.916125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df270c81-61ac-4494-9504-1773dfe5367c None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.916125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 672/1350] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:08:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:55.928440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c12fb-4553-4a7a-bbf6-7b64838fcf0b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.929477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c12fb-4553-4a7a-bbf6-7b64838fcf0b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.929894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c12fb-4553-4a7a-bbf6-7b64838fcf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:56.044328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e16c12fb-4553-4a7a-bbf6-7b64838fcf0b None None] MFA Rules not processed for user `08f34295f19b4d6ea405f1e8c8190db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:56.045775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e16c12fb-4553-4a7a-bbf6-7b64838fcf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:56.066975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:56.172601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 679/1351] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:08:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:00.718738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cbda9-d247-4712-aed6-1c758e46de16 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:00.719235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cbda9-d247-4712-aed6-1c758e46de16 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:00.719317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cbda9-d247-4712-aed6-1c758e46de16 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:00.720845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 673/1352] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:00.725924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc237abb-6dcd-411b-a841-91796943aadf None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:00.726379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc237abb-6dcd-411b-a841-91796943aadf None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:00.726459 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc237abb-6dcd-411b-a841-91796943aadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:00.812397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-fc237abb-6dcd-411b-a841-91796943aadf None None] MFA Rules not processed for user `08f34295f19b4d6ea405f1e8c8190db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:00.813510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-fc237abb-6dcd-411b-a841-91796943aadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:00.908970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:00.952138 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 680/1353] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:09:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:02.337054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a87a2-9318-4d16-818b-59d1c98d8e20 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:02.337585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a87a2-9318-4d16-818b-59d1c98d8e20 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:02.337790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a87a2-9318-4d16-818b-59d1c98d8e20 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:02.339521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 674/1354] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:09:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:02.344947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325ad3-b7c2-4b83-8f9c-09716dc2d7e1 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:02.345438 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325ad3-b7c2-4b83-8f9c-09716dc2d7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:02.345521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325ad3-b7c2-4b83-8f9c-09716dc2d7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:02.430017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d8325ad3-b7c2-4b83-8f9c-09716dc2d7e1 None None] MFA Rules not processed for user `08f34295f19b4d6ea405f1e8c8190db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:02.431274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d8325ad3-b7c2-4b83-8f9c-09716dc2d7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:02.560910 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 681/1355] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:09:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:05.758717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ad1f0-8982-474f-b0a8-d433c8854b41 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:05.759337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ad1f0-8982-474f-b0a8-d433c8854b41 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:05.759440 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ad1f0-8982-474f-b0a8-d433c8854b41 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:05.760866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 675/1356] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:09:05.766276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140fe56-19ab-44be-9b67-ecd30199034e None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:05.766752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140fe56-19ab-44be-9b67-ecd30199034e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:05.766853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140fe56-19ab-44be-9b67-ecd30199034e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:05.804135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:05.855925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-c140fe56-19ab-44be-9b67-ecd30199034e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:05.856732 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c140fe56-19ab-44be-9b67-ecd30199034e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:05.991221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 682/1357] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:06.304083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a742fb-8d2a-4da7-93b8-62f708ce2b73 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:06.304960 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a742fb-8d2a-4da7-93b8-62f708ce2b73 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:06.305777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a742fb-8d2a-4da7-93b8-62f708ce2b73 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:06.308181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 676/1358] 10.222.0.44 () {54 vars in 962 bytes} [Tue Aug 30 01:09:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:06.316646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67ef97-f6a8-4663-ae0e-33bb4da8ac02 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:06.316960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67ef97-f6a8-4663-ae0e-33bb4da8ac02 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:06.317196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67ef97-f6a8-4663-ae0e-33bb4da8ac02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:06.407258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-0d67ef97-f6a8-4663-ae0e-33bb4da8ac02 None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:06.408703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0d67ef97-f6a8-4663-ae0e-33bb4da8ac02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:06.484984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:06.580340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 683/1359] 10.222.0.44 () {58 vars in 1059 bytes} [Tue Aug 30 01:09:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:06.588901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a742fb-8d2a-4da7-93b8-62f708ce2b73 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:06.591822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-18a742fb-8d2a-4da7-93b8-62f708ce2b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:06.767190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a742fb-8d2a-4da7-93b8-62f708ce2b73 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:06.849306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:06.894828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d0cd2-4565-4c54-98b1-527b127f6c39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:06.895445 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d0cd2-4565-4c54-98b1-527b127f6c39 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:06.895723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d0cd2-4565-4c54-98b1-527b127f6c39 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:06.896058 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d0cd2-4565-4c54-98b1-527b127f6c39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:06.897612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-eb1d0cd2-4565-4c54-98b1-527b127f6c39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:07.000121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d0cd2-4565-4c54-98b1-527b127f6c39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:07.001990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d0cd2-4565-4c54-98b1-527b127f6c39 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:07.021795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 677/1360] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:09.220339 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165a74a-64e3-4c16-89eb-6fa036db7578 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:09.220940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165a74a-64e3-4c16-89eb-6fa036db7578 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:09.220940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165a74a-64e3-4c16-89eb-6fa036db7578 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:09.222404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 684/1361] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:09.228492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ef670-f103-427b-9eac-4a733e25964e None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:09.229003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ef670-f103-427b-9eac-4a733e25964e None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:09.229231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ef670-f103-427b-9eac-4a733e25964e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:09.301455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-734ef670-f103-427b-9eac-4a733e25964e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:09.302661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-734ef670-f103-427b-9eac-4a733e25964e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:09.343985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:09.408655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 678/1362] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:09.699243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165a74a-64e3-4c16-89eb-6fa036db7578 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:09.700796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e165a74a-64e3-4c16-89eb-6fa036db7578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:09.891079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165a74a-64e3-4c16-89eb-6fa036db7578 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:09.902563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:10.026215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639e133-03a5-4c67-ae21-3697a5065be8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:10.026901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639e133-03a5-4c67-ae21-3697a5065be8 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:10.027594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639e133-03a5-4c67-ae21-3697a5065be8 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:10.028109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639e133-03a5-4c67-ae21-3697a5065be8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:10.029817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d639e133-03a5-4c67-ae21-3697a5065be8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:10.143835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639e133-03a5-4c67-ae21-3697a5065be8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:10.145548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639e133-03a5-4c67-ae21-3697a5065be8 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:10.166611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 685/1363] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:12.447758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c5275-3e78-4203-bd7b-301a882db7f2 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:12.448857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c5275-3e78-4203-bd7b-301a882db7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:12.449245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c5275-3e78-4203-bd7b-301a882db7f2 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:12.451793 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 679/1364] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:12.457962 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dc256-d201-44da-8479-3d8ce06345f1 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:12.458498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dc256-d201-44da-8479-3d8ce06345f1 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:12.458617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dc256-d201-44da-8479-3d8ce06345f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:12.478069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:12.552659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-bb3dc256-d201-44da-8479-3d8ce06345f1 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:12.553874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-bb3dc256-d201-44da-8479-3d8ce06345f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:12.672095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 686/1365] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:12.969647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1c2c3-3357-483e-8864-7f7bb2a9a4d5 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:12.970010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1c2c3-3357-483e-8864-7f7bb2a9a4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:12.970391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1c2c3-3357-483e-8864-7f7bb2a9a4d5 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:12.971928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 680/1366] 10.222.0.44 () {54 vars in 962 bytes} [Tue Aug 30 01:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:12.978125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d6073-4cb1-44a9-a8bb-07547c8dcfbd None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:12.978714 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d6073-4cb1-44a9-a8bb-07547c8dcfbd None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:12.979079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d6073-4cb1-44a9-a8bb-07547c8dcfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:13.074999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7e0d6073-4cb1-44a9-a8bb-07547c8dcfbd None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:13.076420 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7e0d6073-4cb1-44a9-a8bb-07547c8dcfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:13.120078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:13.241666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 687/1367] 10.222.0.44 () {58 vars in 1059 bytes} [Tue Aug 30 01:09:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:13.250302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1c2c3-3357-483e-8864-7f7bb2a9a4d5 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:13.251854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5cd1c2c3-3357-483e-8864-7f7bb2a9a4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:13.419329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1c2c3-3357-483e-8864-7f7bb2a9a4d5 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:13.443087 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:13.553005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af3c15-e757-40e6-8bd7-702de6da4c80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:13.553700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af3c15-e757-40e6-8bd7-702de6da4c80 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:13.553872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af3c15-e757-40e6-8bd7-702de6da4c80 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:13.554153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af3c15-e757-40e6-8bd7-702de6da4c80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:13.555888 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-03af3c15-e757-40e6-8bd7-702de6da4c80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:13.655032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af3c15-e757-40e6-8bd7-702de6da4c80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:13.656834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af3c15-e757-40e6-8bd7-702de6da4c80 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:13.680611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 681/1368] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:14.310068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe23ea-c968-44f2-87b7-078cd1101159 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:14.310630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe23ea-c968-44f2-87b7-078cd1101159 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:14.310630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe23ea-c968-44f2-87b7-078cd1101159 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:14.312177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 688/1369] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:14.317417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b4f5-6af4-4754-9da9-be5d36819d37 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:14.317703 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b4f5-6af4-4754-9da9-be5d36819d37 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:14.317978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b4f5-6af4-4754-9da9-be5d36819d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:14.355710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:14.396416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-3d19b4f5-6af4-4754-9da9-be5d36819d37 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:14.397626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3d19b4f5-6af4-4754-9da9-be5d36819d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:14.507599 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 682/1370] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:15.189013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf7a8a-f354-4f9f-aa82-8054142db7e3 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:15.189013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf7a8a-f354-4f9f-aa82-8054142db7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:15.189013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf7a8a-f354-4f9f-aa82-8054142db7e3 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:15.192527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 689/1371] 10.222.0.44 () {54 vars in 964 bytes} [Tue Aug 30 01:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:09:15.196480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64182ad0-82fc-4975-8945-6fee455c50a8 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:15.196480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64182ad0-82fc-4975-8945-6fee455c50a8 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:15.196480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64182ad0-82fc-4975-8945-6fee455c50a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:15.289587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-64182ad0-82fc-4975-8945-6fee455c50a8 None None] MFA Rules not processed for user `534da5b47dda43dabaab2cb6aa683342`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:15.290824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-64182ad0-82fc-4975-8945-6fee455c50a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:15.352874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:15.389617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 683/1372] 10.222.0.44 () {58 vars in 1061 bytes} [Tue Aug 30 01:09:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:15.397871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf7a8a-f354-4f9f-aa82-8054142db7e3 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:15.399612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3bbf7a8a-f354-4f9f-aa82-8054142db7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:15.523450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf7a8a-f354-4f9f-aa82-8054142db7e3 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:15.574369 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:15.615467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bad9b3-1d2d-4ddc-b63d-c809078fe049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:15.616112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bad9b3-1d2d-4ddc-b63d-c809078fe049 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:15.616484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bad9b3-1d2d-4ddc-b63d-c809078fe049 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:15.616641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bad9b3-1d2d-4ddc-b63d-c809078fe049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:15.618930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-99bad9b3-1d2d-4ddc-b63d-c809078fe049 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:15.764630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bad9b3-1d2d-4ddc-b63d-c809078fe049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:15.766002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bad9b3-1d2d-4ddc-b63d-c809078fe049 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:15.775725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eac99a-dbbf-4c88-aa1e-82ca446b2802 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:15.776320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eac99a-dbbf-4c88-aa1e-82ca446b2802 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:15.776320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eac99a-dbbf-4c88-aa1e-82ca446b2802 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:15.781772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 684/1373] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:15.782927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acaffd-5e81-4fd6-853b-239e6143ba9d None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:15.783100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acaffd-5e81-4fd6-853b-239e6143ba9d None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:15.783436 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acaffd-5e81-4fd6-853b-239e6143ba9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:15.789654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 690/1374] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:15.862626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-26acaffd-5e81-4fd6-853b-239e6143ba9d None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:15.863834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-26acaffd-5e81-4fd6-853b-239e6143ba9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:15.962729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 685/1375] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:16.259320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bad9b3-1d2d-4ddc-b63d-c809078fe049 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:16.261108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-99bad9b3-1d2d-4ddc-b63d-c809078fe049 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:16.327548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:16.361051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bad9b3-1d2d-4ddc-b63d-c809078fe049 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:16.491383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b41859-24d4-4bb6-b1fd-6c4a084f7e8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:16.492053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b41859-24d4-4bb6-b1fd-6c4a084f7e8d service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:16.492274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b41859-24d4-4bb6-b1fd-6c4a084f7e8d service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:16.492547 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b41859-24d4-4bb6-b1fd-6c4a084f7e8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:16.494202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-70b41859-24d4-4bb6-b1fd-6c4a084f7e8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:16.576979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:16.589261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b41859-24d4-4bb6-b1fd-6c4a084f7e8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:16.590887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b41859-24d4-4bb6-b1fd-6c4a084f7e8d service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:16.610106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 691/1376] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:18.634728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ddfe9-638a-4ff3-b558-469cdd5b79ee None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:18.635258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ddfe9-638a-4ff3-b558-469cdd5b79ee None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:18.635258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ddfe9-638a-4ff3-b558-469cdd5b79ee None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:18.636568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 686/1377] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:18.641869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e4137-4217-456a-93f6-690f892aea39 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:18.642448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e4137-4217-456a-93f6-690f892aea39 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:18.642448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e4137-4217-456a-93f6-690f892aea39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:18.720329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-a23e4137-4217-456a-93f6-690f892aea39 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:18.721610 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a23e4137-4217-456a-93f6-690f892aea39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:18.836510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 692/1378] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:19.138063 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ddfe9-638a-4ff3-b558-469cdd5b79ee None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:19.139599 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f39ddfe9-638a-4ff3-b558-469cdd5b79ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:19.206489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:19.316076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ddfe9-638a-4ff3-b558-469cdd5b79ee None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:19.467433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbda88-81d6-442d-9676-3a3933ceb8aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:19.468054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbda88-81d6-442d-9676-3a3933ceb8aa service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:19.468331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbda88-81d6-442d-9676-3a3933ceb8aa service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:19.468585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbda88-81d6-442d-9676-3a3933ceb8aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:19.470275 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2ebbda88-81d6-442d-9676-3a3933ceb8aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:19.486233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:19.576208 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbda88-81d6-442d-9676-3a3933ceb8aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:19.577812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbda88-81d6-442d-9676-3a3933ceb8aa service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:19.597153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 687/1379] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:09:21.635641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e28c2b-6e15-491c-bbeb-1a3f3ad46edd None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:21.636340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e28c2b-6e15-491c-bbeb-1a3f3ad46edd None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:21.636340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e28c2b-6e15-491c-bbeb-1a3f3ad46edd None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:21.637805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 693/1380] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:21.643656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a6a98-b3a4-4b75-867f-3f5652c02836 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:21.643656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a6a98-b3a4-4b75-867f-3f5652c02836 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:21.644458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a6a98-b3a4-4b75-867f-3f5652c02836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:21.730439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-452a6a98-b3a4-4b75-867f-3f5652c02836 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:21.732404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-452a6a98-b3a4-4b75-867f-3f5652c02836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:21.870427 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:21.888412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 688/1381] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:09:22.202364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e28c2b-6e15-491c-bbeb-1a3f3ad46edd None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:22.204274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d8e28c2b-6e15-491c-bbeb-1a3f3ad46edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:22.284906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:22.395133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e28c2b-6e15-491c-bbeb-1a3f3ad46edd None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:22.533721 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed981b-d235-4515-80ea-9999aeb09453 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:22.534399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed981b-d235-4515-80ea-9999aeb09453 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:22.534703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed981b-d235-4515-80ea-9999aeb09453 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:22.535001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed981b-d235-4515-80ea-9999aeb09453 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:22.536722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-47ed981b-d235-4515-80ea-9999aeb09453 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:22.562940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:22.646654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed981b-d235-4515-80ea-9999aeb09453 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:22.648969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed981b-d235-4515-80ea-9999aeb09453 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:22.671584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 694/1382] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:24.774007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e34c5d-b93e-4494-b0de-74e97be447ad None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:24.774593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e34c5d-b93e-4494-b0de-74e97be447ad None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:24.774593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e34c5d-b93e-4494-b0de-74e97be447ad None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:24.776113 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 689/1383] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:24.781839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c760fe7-be80-4458-b1be-188f476b9d21 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:24.782351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c760fe7-be80-4458-b1be-188f476b9d21 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:24.782458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c760fe7-be80-4458-b1be-188f476b9d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:24.852187 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-9c760fe7-be80-4458-b1be-188f476b9d21 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:24.854516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9c760fe7-be80-4458-b1be-188f476b9d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:24.970742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:24.975828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 695/1384] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:25.393926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e34c5d-b93e-4494-b0de-74e97be447ad None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:25.396320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-22e34c5d-b93e-4494-b0de-74e97be447ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:25.570063 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e34c5d-b93e-4494-b0de-74e97be447ad None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:25.663963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:25.729729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766d2d9-8c4b-43db-8d4c-da2adcbc6f8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:25.730332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766d2d9-8c4b-43db-8d4c-da2adcbc6f8f service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:25.730609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766d2d9-8c4b-43db-8d4c-da2adcbc6f8f service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:25.730950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766d2d9-8c4b-43db-8d4c-da2adcbc6f8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:25.732510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3766d2d9-8c4b-43db-8d4c-da2adcbc6f8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:25.831910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766d2d9-8c4b-43db-8d4c-da2adcbc6f8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:25.833570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766d2d9-8c4b-43db-8d4c-da2adcbc6f8f service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:25.855318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 690/1385] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:28.217171 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3ffea-3825-419e-a35f-5c618184b67d None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:28.217782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3ffea-3825-419e-a35f-5c618184b67d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:28.217782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3ffea-3825-419e-a35f-5c618184b67d None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:28.219162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 696/1386] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:28.225611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634011e-40ca-4812-9929-0f02f142e20a None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:28.226105 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634011e-40ca-4812-9929-0f02f142e20a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:28.226217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634011e-40ca-4812-9929-0f02f142e20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:28.306070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-5634011e-40ca-4812-9929-0f02f142e20a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:28.307595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5634011e-40ca-4812-9929-0f02f142e20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:28.343187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:28.423268 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 691/1387] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:28.828468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3ffea-3825-419e-a35f-5c618184b67d None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:28.830435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-fdd3ffea-3825-419e-a35f-5c618184b67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:29.016399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3ffea-3825-419e-a35f-5c618184b67d None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:29.100700 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:29.154074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a393fa-4a5e-467f-84aa-fd23e9a2484c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:29.154689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a393fa-4a5e-467f-84aa-fd23e9a2484c service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:29.154966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a393fa-4a5e-467f-84aa-fd23e9a2484c service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:29.155238 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a393fa-4a5e-467f-84aa-fd23e9a2484c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:29.156917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-82a393fa-4a5e-467f-84aa-fd23e9a2484c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:29.252793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a393fa-4a5e-467f-84aa-fd23e9a2484c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:29.254431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a393fa-4a5e-467f-84aa-fd23e9a2484c service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:29.277389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 697/1388] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:31.196702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fe050-70a7-4bee-9c70-0653ea014021 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:31.197365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fe050-70a7-4bee-9c70-0653ea014021 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:31.197365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fe050-70a7-4bee-9c70-0653ea014021 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:31.198656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 692/1389] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:31.204535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f66f0-5681-439f-9322-e54069e33206 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:31.205104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f66f0-5681-439f-9322-e54069e33206 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:31.205104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f66f0-5681-439f-9322-e54069e33206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:31.276011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-802f66f0-5681-439f-9322-e54069e33206 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:31.277259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-802f66f0-5681-439f-9322-e54069e33206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:31.312894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:31.326876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de656e-c1f6-4a1b-a35c-dd88014b7ec8 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:31.327159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de656e-c1f6-4a1b-a35c-dd88014b7ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:31.327432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de656e-c1f6-4a1b-a35c-dd88014b7ec8 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:31.328839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 693/1390] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:09:31.334281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7af9ce-55c8-4e24-a097-11cb3f3044a9 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:31.334562 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7af9ce-55c8-4e24-a097-11cb3f3044a9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:31.334838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7af9ce-55c8-4e24-a097-11cb3f3044a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:31.379617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 698/1391] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:31.406170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-0f7af9ce-55c8-4e24-a097-11cb3f3044a9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:31.407385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0f7af9ce-55c8-4e24-a097-11cb3f3044a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:31.520797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 694/1392] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:09:31.711599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c15b7-6e33-4e1b-a209-79ff868c5547 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:31.711880 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c15b7-6e33-4e1b-a209-79ff868c5547 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:31.711973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c15b7-6e33-4e1b-a209-79ff868c5547 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:31.714080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 699/1393] 10.222.0.44 () {54 vars in 967 bytes} [Tue Aug 30 01:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:31.720093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae79885-4071-473d-8e22-bb877463821f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:31.720462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae79885-4071-473d-8e22-bb877463821f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:31.720805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae79885-4071-473d-8e22-bb877463821f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:31.731644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:31.814739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-eae79885-4071-473d-8e22-bb877463821f None None] MFA Rules not processed for user `92c9d444b3294ed18c3d488cf0a3eeb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:31.815980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-eae79885-4071-473d-8e22-bb877463821f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:31.818066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c15b7-6e33-4e1b-a209-79ff868c5547 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:31.819676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-205c15b7-6e33-4e1b-a209-79ff868c5547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:31.907356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 695/1394] 10.222.0.44 () {58 vars in 1064 bytes} [Tue Aug 30 01:09:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:31.914991 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae79885-4071-473d-8e22-bb877463821f None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:31.923166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae79885-4071-473d-8e22-bb877463821f None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:31.980982 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c15b7-6e33-4e1b-a209-79ff868c5547 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:32.002409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56903eb3-0778-4375-b688-987d50bedb37 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c9d444b3294ed18c3d488cf0a3eeb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:32.002979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:32.003124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56903eb3-0778-4375-b688-987d50bedb37 service glance] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:32.003280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56903eb3-0778-4375-b688-987d50bedb37 service glance] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:32.003567 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56903eb3-0778-4375-b688-987d50bedb37 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:32.005352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-56903eb3-0778-4375-b688-987d50bedb37 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:32.031560 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:32.121398 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf009bec-0316-40f6-901a-dfd4aa37e9b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:32.121931 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf009bec-0316-40f6-901a-dfd4aa37e9b6 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:32.122208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf009bec-0316-40f6-901a-dfd4aa37e9b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:32.122482 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf009bec-0316-40f6-901a-dfd4aa37e9b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:32.124189 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-cf009bec-0316-40f6-901a-dfd4aa37e9b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:32.138349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56903eb3-0778-4375-b688-987d50bedb37 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:32.139830 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56903eb3-0778-4375-b688-987d50bedb37 service glance] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:32.161161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 696/1395] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:09:32.232638 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf009bec-0316-40f6-901a-dfd4aa37e9b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:32.234470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf009bec-0316-40f6-901a-dfd4aa37e9b6 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:32.255799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 700/1396] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:32.969349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b53fa9-4118-4f02-b72b-4bf62a214b4f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:32.970203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b53fa9-4118-4f02-b72b-4bf62a214b4f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:32.970203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b53fa9-4118-4f02-b72b-4bf62a214b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:33.054815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-89b53fa9-4118-4f02-b72b-4bf62a214b4f None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:33.056816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-89b53fa9-4118-4f02-b72b-4bf62a214b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:33.140457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:33.155021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 697/1397] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:09:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:09:33.203345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de6b06-3018-40f1-9e08-78a974a466fd None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:33.203746 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de6b06-3018-40f1-9e08-78a974a466fd None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:33.204247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de6b06-3018-40f1-9e08-78a974a466fd None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:33.205903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 701/1398] 10.222.0.44 () {54 vars in 943 bytes} [Tue Aug 30 01:09:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:33.212649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f1e0f-02bb-466a-82ac-c86940c2a2f6 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:33.213064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f1e0f-02bb-466a-82ac-c86940c2a2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:33.213870 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f1e0f-02bb-466a-82ac-c86940c2a2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:33.299748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-137f1e0f-02bb-466a-82ac-c86940c2a2f6 None None] MFA Rules not processed for user `3330df65b64c4fffbc3bb687d5c51021`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:33.301122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-137f1e0f-02bb-466a-82ac-c86940c2a2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:33.427300 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 698/1399] 10.222.0.44 () {58 vars in 1040 bytes} [Tue Aug 30 01:09:33 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:33.437429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de6b06-3018-40f1-9e08-78a974a466fd None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:33.439032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-42de6b06-3018-40f1-9e08-78a974a466fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:33.466826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:33.626961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de6b06-3018-40f1-9e08-78a974a466fd None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:33.743575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15256560-eeff-42a2-9b86-3366d6094d05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:33.744244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15256560-eeff-42a2-9b86-3366d6094d05 service swift] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:33.745492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15256560-eeff-42a2-9b86-3366d6094d05 service swift] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:33.745695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15256560-eeff-42a2-9b86-3366d6094d05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:33.747380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-15256560-eeff-42a2-9b86-3366d6094d05 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:33.755172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:33.844324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15256560-eeff-42a2-9b86-3366d6094d05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:33.846934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15256560-eeff-42a2-9b86-3366d6094d05 service swift] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:33.856350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 702/1400] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:09:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:09:34.269206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ec75d-d6f4-4230-b12f-a697e530a3a2 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:34.269206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ec75d-d6f4-4230-b12f-a697e530a3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:34.269206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ec75d-d6f4-4230-b12f-a697e530a3a2 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:34.271482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 699/1401] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:34.277810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aed236-d571-4ad4-9803-98176093c861 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:34.278090 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aed236-d571-4ad4-9803-98176093c861 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:34.278362 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aed236-d571-4ad4-9803-98176093c861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:34.362318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-93aed236-d571-4ad4-9803-98176093c861 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:34.363985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-93aed236-d571-4ad4-9803-98176093c861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:34.541641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:34.576308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 703/1402] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:34.867529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ec75d-d6f4-4230-b12f-a697e530a3a2 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:34.869269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-539ec75d-d6f4-4230-b12f-a697e530a3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:34.939845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:35.054350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ec75d-d6f4-4230-b12f-a697e530a3a2 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:35.212110 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f97db2-aa90-4186-adeb-e78217ff0112 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:35.213012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f97db2-aa90-4186-adeb-e78217ff0112 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:35.213974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f97db2-aa90-4186-adeb-e78217ff0112 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:35.214326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f97db2-aa90-4186-adeb-e78217ff0112 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:35.216034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d0f97db2-aa90-4186-adeb-e78217ff0112 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:35.225212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:35.336185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f97db2-aa90-4186-adeb-e78217ff0112 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:35.337879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f97db2-aa90-4186-adeb-e78217ff0112 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:35.362107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 700/1403] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:37.371837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8cf42-f877-4a7c-8ca4-f842ab9413a4 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:37.372459 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8cf42-f877-4a7c-8ca4-f842ab9413a4 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:37.372459 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8cf42-f877-4a7c-8ca4-f842ab9413a4 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:37.373766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 704/1404] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:37.379273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fbb6d-747a-4aab-83fb-998e563e7210 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:37.380037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fbb6d-747a-4aab-83fb-998e563e7210 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:37.380495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fbb6d-747a-4aab-83fb-998e563e7210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:37.459707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-3b7fbb6d-747a-4aab-83fb-998e563e7210 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:37.461031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3b7fbb6d-747a-4aab-83fb-998e563e7210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:37.554420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:37.579328 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 701/1405] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:37.877927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8cf42-f877-4a7c-8ca4-f842ab9413a4 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:37.879463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-fef8cf42-f877-4a7c-8ca4-f842ab9413a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:38.082421 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8cf42-f877-4a7c-8ca4-f842ab9413a4 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:38.169289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:38.215283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0a6e3-cabb-408f-a2ff-d4056252249d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:38.215885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0a6e3-cabb-408f-a2ff-d4056252249d service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:38.216206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0a6e3-cabb-408f-a2ff-d4056252249d service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:38.216474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0a6e3-cabb-408f-a2ff-d4056252249d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:38.218194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8ee0a6e3-cabb-408f-a2ff-d4056252249d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:38.320474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0a6e3-cabb-408f-a2ff-d4056252249d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:38.322196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0a6e3-cabb-408f-a2ff-d4056252249d service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:38.350341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 705/1406] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:40.503365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047adf3-5425-4711-a63a-f8f4cd2466b6 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:40.503946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047adf3-5425-4711-a63a-f8f4cd2466b6 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:40.503946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047adf3-5425-4711-a63a-f8f4cd2466b6 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:40.505264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 702/1407] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:40.511568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c7f85-4bca-403d-b588-f2ad64c58cc1 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:40.512263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c7f85-4bca-403d-b588-f2ad64c58cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:40.512263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c7f85-4bca-403d-b588-f2ad64c58cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:40.592327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ce0c7f85-4bca-403d-b588-f2ad64c58cc1 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:40.593651 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ce0c7f85-4bca-403d-b588-f2ad64c58cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:40.636198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:40.713984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 706/1408] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:09:41.021395 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047adf3-5425-4711-a63a-f8f4cd2466b6 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:41.023190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b047adf3-5425-4711-a63a-f8f4cd2466b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:41.206928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047adf3-5425-4711-a63a-f8f4cd2466b6 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:41.276160 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:41.336157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c20ec4-7df4-4eee-80df-6ea7cce6a490 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:41.336810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c20ec4-7df4-4eee-80df-6ea7cce6a490 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:41.337091 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c20ec4-7df4-4eee-80df-6ea7cce6a490 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:41.337363 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c20ec4-7df4-4eee-80df-6ea7cce6a490 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:41.339055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-97c20ec4-7df4-4eee-80df-6ea7cce6a490 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:41.433521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c20ec4-7df4-4eee-80df-6ea7cce6a490 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:41.435225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c20ec4-7df4-4eee-80df-6ea7cce6a490 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:41.455748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 703/1409] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:50.710822 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=27 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:51.453459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=68 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:56.415795 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c916d6-28a8-4926-9107-976450a29924 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:56.416400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c916d6-28a8-4926-9107-976450a29924 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:56.416400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c916d6-28a8-4926-9107-976450a29924 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:56.417822 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 707/1410] 10.222.0.44 () {54 vars in 918 bytes} [Tue Aug 30 01:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:56.422844 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e2392-2678-4cee-9015-4e1fdf202835 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:56.423280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e2392-2678-4cee-9015-4e1fdf202835 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:56.423376 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e2392-2678-4cee-9015-4e1fdf202835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:56.502963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d30e2392-2678-4cee-9015-4e1fdf202835 None None] MFA Rules not processed for user `08f34295f19b4d6ea405f1e8c8190db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:56.504210 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d30e2392-2678-4cee-9015-4e1fdf202835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:56.611207 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 704/1411] 10.222.0.44 () {58 vars in 1015 bytes} [Tue Aug 30 01:09:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:56.626206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d46619-ff13-4ea3-a9aa-842ab91ab943 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:56.626492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d46619-ff13-4ea3-a9aa-842ab91ab943 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:56.626764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d46619-ff13-4ea3-a9aa-842ab91ab943 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:56.628202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 708/1412] 10.222.0.44 () {54 vars in 918 bytes} [Tue Aug 30 01:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:56.632797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a888ee4-dc10-46e5-9730-5f68f6578f8b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:56.633076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a888ee4-dc10-46e5-9730-5f68f6578f8b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:56.633348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a888ee4-dc10-46e5-9730-5f68f6578f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:56.708269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:56.711120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-1a888ee4-dc10-46e5-9730-5f68f6578f8b None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:56.712926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1a888ee4-dc10-46e5-9730-5f68f6578f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:56.832731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 705/1413] 10.222.0.44 () {58 vars in 1015 bytes} [Tue Aug 30 01:09:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:59.236097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ad6d8-5692-4c32-9f73-56c621163104 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:59.236694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ad6d8-5692-4c32-9f73-56c621163104 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:59.236694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ad6d8-5692-4c32-9f73-56c621163104 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:59.237982 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:59.243121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b584bc-bb21-44f0-a045-2fac6fe34f7f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:59.243568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b584bc-bb21-44f0-a045-2fac6fe34f7f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:59.243674 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b584bc-bb21-44f0-a045-2fac6fe34f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:59.313373 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-e0b584bc-bb21-44f0-a045-2fac6fe34f7f None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:59.314672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e0b584bc-bb21-44f0-a045-2fac6fe34f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:59.372010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:59.414977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:59.705325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ad6d8-5692-4c32-9f73-56c621163104 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:59.710405 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-825ad6d8-5692-4c32-9f73-56c621163104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:59.852058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ad6d8-5692-4c32-9f73-56c621163104 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:59.928483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:59.969421 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ba91a-4f49-4653-b341-0e10149b2f5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:59.970025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ba91a-4f49-4653-b341-0e10149b2f5e service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:59.970294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ba91a-4f49-4653-b341-0e10149b2f5e service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:59.970558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ba91a-4f49-4653-b341-0e10149b2f5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:59.972169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-524ba91a-4f49-4653-b341-0e10149b2f5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:00.060710 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ba91a-4f49-4653-b341-0e10149b2f5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:00.062345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ba91a-4f49-4653-b341-0e10149b2f5e service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:00.080052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 710/1416] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:00.974842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccb444-9a8f-446f-8c3f-ce0b794e4271 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:00.975355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccb444-9a8f-446f-8c3f-ce0b794e4271 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:00.975355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccb444-9a8f-446f-8c3f-ce0b794e4271 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:00.976727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:00.982064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57661c6-bca2-4077-a073-e8ae2cd6d88f None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:00.982494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57661c6-bca2-4077-a073-e8ae2cd6d88f None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:00.982606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57661c6-bca2-4077-a073-e8ae2cd6d88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:01.061900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d57661c6-bca2-4077-a073-e8ae2cd6d88f None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:01.063152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d57661c6-bca2-4077-a073-e8ae2cd6d88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:01.090852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:01.164012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:01.315799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccb444-9a8f-446f-8c3f-ce0b794e4271 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:01.317278 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9bccb444-9a8f-446f-8c3f-ce0b794e4271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:01.467629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bccb444-9a8f-446f-8c3f-ce0b794e4271 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:01.510169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:01.585867 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e82674-16ee-4039-8066-0330409821b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:01.586386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e82674-16ee-4039-8066-0330409821b1 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:01.586667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e82674-16ee-4039-8066-0330409821b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:01.586937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e82674-16ee-4039-8066-0330409821b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:01.588574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-63e82674-16ee-4039-8066-0330409821b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:01.678682 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e82674-16ee-4039-8066-0330409821b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:01.680314 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e82674-16ee-4039-8066-0330409821b1 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:01.699533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 708/1419] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:02.616466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2faaa-e160-4d28-8519-0289df471853 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:02.617028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2faaa-e160-4d28-8519-0289df471853 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:02.617028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2faaa-e160-4d28-8519-0289df471853 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:02.618405 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:02.623805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86a090-5267-451f-8f06-ef8f350e2e22 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:02.624273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86a090-5267-451f-8f06-ef8f350e2e22 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:02.624380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86a090-5267-451f-8f06-ef8f350e2e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:02.680172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:02.704233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8c86a090-5267-451f-8f06-ef8f350e2e22 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:02.705438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8c86a090-5267-451f-8f06-ef8f350e2e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:02.802389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:02.958487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2faaa-e160-4d28-8519-0289df471853 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:02.960232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-40c2faaa-e160-4d28-8519-0289df471853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:03.111002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2faaa-e160-4d28-8519-0289df471853 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:03.123792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:03.232564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59a465-a7ed-4fe4-95e0-6f88a6d6453f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:03.233172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59a465-a7ed-4fe4-95e0-6f88a6d6453f service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:03.233451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59a465-a7ed-4fe4-95e0-6f88a6d6453f service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:03.233722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59a465-a7ed-4fe4-95e0-6f88a6d6453f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:03.235322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-cf59a465-a7ed-4fe4-95e0-6f88a6d6453f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:03.324373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59a465-a7ed-4fe4-95e0-6f88a6d6453f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:03.325985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59a465-a7ed-4fe4-95e0-6f88a6d6453f service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:03.343502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 713/1422] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:04.209979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bd760-71da-47a4-822d-020c1542b318 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:04.210553 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bd760-71da-47a4-822d-020c1542b318 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:04.210553 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bd760-71da-47a4-822d-020c1542b318 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:04.211874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:04.217272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7f24d-8b4e-4552-a39c-d64ed8a471fb None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:04.217708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7f24d-8b4e-4552-a39c-d64ed8a471fb None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:04.217813 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7f24d-8b4e-4552-a39c-d64ed8a471fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:04.237284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:04.295945 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6ec7f24d-8b4e-4552-a39c-d64ed8a471fb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:04.297222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6ec7f24d-8b4e-4552-a39c-d64ed8a471fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:04.398836 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:04.551336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bd760-71da-47a4-822d-020c1542b318 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:04.552797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b71bd760-71da-47a4-822d-020c1542b318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:04.665732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:04.703186 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bd760-71da-47a4-822d-020c1542b318 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:04.816955 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b567f8d-f619-49d3-9c39-88603b94e59b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:04.817475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b567f8d-f619-49d3-9c39-88603b94e59b service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:04.817748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b567f8d-f619-49d3-9c39-88603b94e59b service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:04.818012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b567f8d-f619-49d3-9c39-88603b94e59b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:04.819595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8b567f8d-f619-49d3-9c39-88603b94e59b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:04.895198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:04.909475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b567f8d-f619-49d3-9c39-88603b94e59b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:04.911074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b567f8d-f619-49d3-9c39-88603b94e59b service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:04.929642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 711/1425] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:05.806189 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab71f0-7c52-47a8-949c-da248f4de95d None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:05.806752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab71f0-7c52-47a8-949c-da248f4de95d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:05.806752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab71f0-7c52-47a8-949c-da248f4de95d None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:05.808641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:10:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:05.813707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b2690-576a-44e7-9191-3ae2b2ea95df None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:05.814134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b2690-576a-44e7-9191-3ae2b2ea95df None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:05.814265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b2690-576a-44e7-9191-3ae2b2ea95df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:05.890622 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-e03b2690-576a-44e7-9191-3ae2b2ea95df None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:05.891861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e03b2690-576a-44e7-9191-3ae2b2ea95df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:05.987261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:06.142259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab71f0-7c52-47a8-949c-da248f4de95d None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:06.143773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d6ab71f0-7c52-47a8-949c-da248f4de95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:06.229130 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:06.291548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab71f0-7c52-47a8-949c-da248f4de95d None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:06.405351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d3d8c-dff1-4d67-940b-c78fe6e66a98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:06.405979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d3d8c-dff1-4d67-940b-c78fe6e66a98 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:06.406259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d3d8c-dff1-4d67-940b-c78fe6e66a98 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:06.406528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d3d8c-dff1-4d67-940b-c78fe6e66a98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:06.408158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f58d3d8c-dff1-4d67-940b-c78fe6e66a98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:06.457047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:06.501695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d3d8c-dff1-4d67-940b-c78fe6e66a98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:06.503227 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d3d8c-dff1-4d67-940b-c78fe6e66a98 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:06.520969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 716/1428] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:07.340377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d0546-588e-4a36-ac13-a3769298db31 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:07.340959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d0546-588e-4a36-ac13-a3769298db31 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:07.340959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d0546-588e-4a36-ac13-a3769298db31 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:07.342311 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:07.347308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91378272-da73-43ff-89e8-43f8bfcebfdb None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:07.347739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91378272-da73-43ff-89e8-43f8bfcebfdb None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:07.347875 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91378272-da73-43ff-89e8-43f8bfcebfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:07.423140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-91378272-da73-43ff-89e8-43f8bfcebfdb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:07.424391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-91378272-da73-43ff-89e8-43f8bfcebfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:07.521618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:07.677585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d0546-588e-4a36-ac13-a3769298db31 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:07.679096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9e7d0546-588e-4a36-ac13-a3769298db31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:07.728399 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:07.826558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d0546-588e-4a36-ac13-a3769298db31 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:07.946169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d6ee7-5578-419b-818f-2e31644b2de9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:07.946777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d6ee7-5578-419b-818f-2e31644b2de9 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:07.947053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d6ee7-5578-419b-818f-2e31644b2de9 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:07.947322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d6ee7-5578-419b-818f-2e31644b2de9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:07.949118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0a1d6ee7-5578-419b-818f-2e31644b2de9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:07.956153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:08.069385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d6ee7-5578-419b-818f-2e31644b2de9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:08.070969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d6ee7-5578-419b-818f-2e31644b2de9 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:08.091454 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 714/1431] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:08.917432 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76df1f-0d30-4955-b2df-5c6565050eae None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:08.918151 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76df1f-0d30-4955-b2df-5c6565050eae None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:08.918151 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76df1f-0d30-4955-b2df-5c6565050eae None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:08.919350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:10:08.924983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c625a9-bcff-4aa2-962d-6c87d52bb303 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:08.925608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c625a9-bcff-4aa2-962d-6c87d52bb303 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:08.925608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c625a9-bcff-4aa2-962d-6c87d52bb303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:09.007969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-03c625a9-bcff-4aa2-962d-6c87d52bb303 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:09.009129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-03c625a9-bcff-4aa2-962d-6c87d52bb303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:09.087750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:09.121894 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:09.295939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76df1f-0d30-4955-b2df-5c6565050eae None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:09.297446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-eb76df1f-0d30-4955-b2df-5c6565050eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:09.318175 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:09.466836 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76df1f-0d30-4955-b2df-5c6565050eae None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:09.565900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:09.593850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dea5b-2b7a-4d67-855d-79ca24e19fcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:09.594485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dea5b-2b7a-4d67-855d-79ca24e19fcf service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:09.594766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dea5b-2b7a-4d67-855d-79ca24e19fcf service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:09.595046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dea5b-2b7a-4d67-855d-79ca24e19fcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:09.596793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a57dea5b-2b7a-4d67-855d-79ca24e19fcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:09.690021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57dea5b-2b7a-4d67-855d-79ca24e19fcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:09.691647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57dea5b-2b7a-4d67-855d-79ca24e19fcf service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:09.709659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 719/1434] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:10.534533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60296a58-7307-4921-9554-8965885359f9 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:10.535084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60296a58-7307-4921-9554-8965885359f9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:10.535084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60296a58-7307-4921-9554-8965885359f9 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:10.536535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:10.541559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae0d6e-c963-42e2-8e2c-96d3f0ee5fd5 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:10.542014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae0d6e-c963-42e2-8e2c-96d3f0ee5fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:10.542144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae0d6e-c963-42e2-8e2c-96d3f0ee5fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:10.626585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-2eae0d6e-c963-42e2-8e2c-96d3f0ee5fd5 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:10.627887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2eae0d6e-c963-42e2-8e2c-96d3f0ee5fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:10.689846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:10.748568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:10.903412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60296a58-7307-4921-9554-8965885359f9 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:10.904891 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-60296a58-7307-4921-9554-8965885359f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:11.050657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60296a58-7307-4921-9554-8965885359f9 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:11.114774 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:11.177516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797128e-9648-431a-b760-7a2eafb487f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:11.178127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797128e-9648-431a-b760-7a2eafb487f2 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:11.178404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797128e-9648-431a-b760-7a2eafb487f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:11.178675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797128e-9648-431a-b760-7a2eafb487f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:11.180389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0797128e-9648-431a-b760-7a2eafb487f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:11.273144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797128e-9648-431a-b760-7a2eafb487f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:11.274738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797128e-9648-431a-b760-7a2eafb487f2 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:11.294885 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 717/1437] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:10:12.130730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de736d2-328c-446c-b852-965a3627b771 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:12.131319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de736d2-328c-446c-b852-965a3627b771 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:12.131425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de736d2-328c-446c-b852-965a3627b771 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:12.132738 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:12.138011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c10c88-7314-4d9f-b6f6-82ccb3913d6d None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:12.138464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c10c88-7314-4d9f-b6f6-82ccb3913d6d None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:12.138575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c10c88-7314-4d9f-b6f6-82ccb3913d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:12.219894 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d0c10c88-7314-4d9f-b6f6-82ccb3913d6d None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:12.221141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d0c10c88-7314-4d9f-b6f6-82ccb3913d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:12.223184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:12.324727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:12.482766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de736d2-328c-446c-b852-965a3627b771 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:12.485415 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7de736d2-328c-446c-b852-965a3627b771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:12.663314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de736d2-328c-446c-b852-965a3627b771 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:12.698127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:12.783695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a919b-3c51-49bb-9ca2-56ccfbbd27c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:12.784254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a919b-3c51-49bb-9ca2-56ccfbbd27c0 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:12.784531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a919b-3c51-49bb-9ca2-56ccfbbd27c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:12.784804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a919b-3c51-49bb-9ca2-56ccfbbd27c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:12.786602 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-365a919b-3c51-49bb-9ca2-56ccfbbd27c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:12.884661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a919b-3c51-49bb-9ca2-56ccfbbd27c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:12.886607 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a919b-3c51-49bb-9ca2-56ccfbbd27c0 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:12.906438 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 722/1440] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:10:14.783097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8885d2-5cee-4b96-89c5-a78070819e7d None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:14.783670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8885d2-5cee-4b96-89c5-a78070819e7d None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:14.783670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8885d2-5cee-4b96-89c5-a78070819e7d None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:14.785030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:14.790494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be819220-edf0-47e5-811f-d310239bebc2 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:14.790921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be819220-edf0-47e5-811f-d310239bebc2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:14.791076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be819220-edf0-47e5-811f-d310239bebc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:14.831721 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:14.858980 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-be819220-edf0-47e5-811f-d310239bebc2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:14.860208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-be819220-edf0-47e5-811f-d310239bebc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:14.966309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:10:15.257692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8885d2-5cee-4b96-89c5-a78070819e7d None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:15.259321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6e8885d2-5cee-4b96-89c5-a78070819e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:15.368939 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8885d2-5cee-4b96-89c5-a78070819e7d None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:15.401517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:15.444746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a8e82-10ec-4eaf-8632-a94cc21140f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:15.445377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a8e82-10ec-4eaf-8632-a94cc21140f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:15.445682 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a8e82-10ec-4eaf-8632-a94cc21140f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:15.445975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a8e82-10ec-4eaf-8632-a94cc21140f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:15.447577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-140a8e82-10ec-4eaf-8632-a94cc21140f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:15.559218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a8e82-10ec-4eaf-8632-a94cc21140f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:15.560623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a8e82-10ec-4eaf-8632-a94cc21140f1 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:15.579069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 720/1443] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:17.780223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426470f0-03ef-4a42-ad5a-ad6aee067365 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:17.780812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426470f0-03ef-4a42-ad5a-ad6aee067365 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:17.780812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426470f0-03ef-4a42-ad5a-ad6aee067365 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:17.782113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:17.787468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fd44d-f388-43e4-918f-93c6f86d2efb None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:17.787920 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fd44d-f388-43e4-918f-93c6f86d2efb None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:17.788036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fd44d-f388-43e4-918f-93c6f86d2efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:17.854251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-5f2fd44d-f388-43e4-918f-93c6f86d2efb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:17.856144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5f2fd44d-f388-43e4-918f-93c6f86d2efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:17.868949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:17.959252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:18.255196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426470f0-03ef-4a42-ad5a-ad6aee067365 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:18.256795 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-426470f0-03ef-4a42-ad5a-ad6aee067365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:18.359835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426470f0-03ef-4a42-ad5a-ad6aee067365 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:18.372367 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:18.454573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c6743-b777-4d11-a4f4-825f6ab82d3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:18.455187 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c6743-b777-4d11-a4f4-825f6ab82d3d service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:18.455467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c6743-b777-4d11-a4f4-825f6ab82d3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:18.455738 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c6743-b777-4d11-a4f4-825f6ab82d3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:18.457375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f59c6743-b777-4d11-a4f4-825f6ab82d3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:18.569266 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c6743-b777-4d11-a4f4-825f6ab82d3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:18.570641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c6743-b777-4d11-a4f4-825f6ab82d3d service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:18.588614 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 725/1446] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:20.820411 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085e572-93b3-48d0-bb31-43a17963f67a None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:20.820944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085e572-93b3-48d0-bb31-43a17963f67a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:20.820944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085e572-93b3-48d0-bb31-43a17963f67a None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:20.822341 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:10:20.828385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48608d1a-14cc-4623-907d-0acc4d83e033 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:20.828814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48608d1a-14cc-4623-907d-0acc4d83e033 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:20.828947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48608d1a-14cc-4623-907d-0acc4d83e033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:20.877969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:20.896247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-48608d1a-14cc-4623-907d-0acc4d83e033 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:20.897454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-48608d1a-14cc-4623-907d-0acc4d83e033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:21.002262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:21.296343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085e572-93b3-48d0-bb31-43a17963f67a None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:21.297918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9085e572-93b3-48d0-bb31-43a17963f67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:21.401127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085e572-93b3-48d0-bb31-43a17963f67a None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:21.442442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:21.476817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dde1a-d786-4cb6-9da5-a412c4f91257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:21.477416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dde1a-d786-4cb6-9da5-a412c4f91257 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:21.477685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dde1a-d786-4cb6-9da5-a412c4f91257 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:21.477946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dde1a-d786-4cb6-9da5-a412c4f91257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:21.479522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9c3dde1a-d786-4cb6-9da5-a412c4f91257 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:21.592078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3dde1a-d786-4cb6-9da5-a412c4f91257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:21.593415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3dde1a-d786-4cb6-9da5-a412c4f91257 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:21.612372 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 723/1449] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:23.696922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd39458-0064-49eb-b906-abf740f76b30 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:23.697448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd39458-0064-49eb-b906-abf740f76b30 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:23.697448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd39458-0064-49eb-b906-abf740f76b30 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:23.699425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:10:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:23.704937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66de518-9a73-405c-87bd-dbab7555fcbb None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:23.705375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66de518-9a73-405c-87bd-dbab7555fcbb None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:23.705498 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66de518-9a73-405c-87bd-dbab7555fcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:23.771643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c66de518-9a73-405c-87bd-dbab7555fcbb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:23.773130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c66de518-9a73-405c-87bd-dbab7555fcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:23.793421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:23.872147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:24.156726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd39458-0064-49eb-b906-abf740f76b30 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:24.158344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3bd39458-0064-49eb-b906-abf740f76b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:24.262133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd39458-0064-49eb-b906-abf740f76b30 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:24.281783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:24.334725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f46630-cf32-422f-8be1-e1b5f2746768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:24.335329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f46630-cf32-422f-8be1-e1b5f2746768 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:24.335603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f46630-cf32-422f-8be1-e1b5f2746768 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:24.335868 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f46630-cf32-422f-8be1-e1b5f2746768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:24.337486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-18f46630-cf32-422f-8be1-e1b5f2746768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:24.446432 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f46630-cf32-422f-8be1-e1b5f2746768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:24.447869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f46630-cf32-422f-8be1-e1b5f2746768 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:24.466509 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 728/1452] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:26.493566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a499e8-5969-4b47-98f8-d876614b1b83 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:26.494163 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a499e8-5969-4b47-98f8-d876614b1b83 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:26.494276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a499e8-5969-4b47-98f8-d876614b1b83 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:26.495583 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:26.500931 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b0ea3-0d3c-470c-b9c0-62207752eaea None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:26.501388 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b0ea3-0d3c-470c-b9c0-62207752eaea None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:26.501494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b0ea3-0d3c-470c-b9c0-62207752eaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:26.567977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:26.571158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-bd9b0ea3-0d3c-470c-b9c0-62207752eaea None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:26.573010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-bd9b0ea3-0d3c-470c-b9c0-62207752eaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:26.668038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:26.954088 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a499e8-5969-4b47-98f8-d876614b1b83 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:26.955626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-30a499e8-5969-4b47-98f8-d876614b1b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:27.063390 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a499e8-5969-4b47-98f8-d876614b1b83 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:27.114028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:27.139703 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b0e19-15e0-4ec1-9046-d461d4f4b607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:27.140348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b0e19-15e0-4ec1-9046-d461d4f4b607 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:27.140631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b0e19-15e0-4ec1-9046-d461d4f4b607 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:27.140904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b0e19-15e0-4ec1-9046-d461d4f4b607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:27.142552 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-441b0e19-15e0-4ec1-9046-d461d4f4b607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:27.263710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b0e19-15e0-4ec1-9046-d461d4f4b607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:27.265087 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b0e19-15e0-4ec1-9046-d461d4f4b607 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:27.283129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 726/1455] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:29.239809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d3908-49c1-49ba-82e8-cbeb66b70591 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:29.240445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d3908-49c1-49ba-82e8-cbeb66b70591 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:29.240445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d3908-49c1-49ba-82e8-cbeb66b70591 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:29.241743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:29.247476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9337f9-3b0e-4c3d-bb92-fd5aec6c3f1b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:29.247884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9337f9-3b0e-4c3d-bb92-fd5aec6c3f1b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:29.248016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9337f9-3b0e-4c3d-bb92-fd5aec6c3f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:29.315108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-aa9337f9-3b0e-4c3d-bb92-fd5aec6c3f1b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:29.316335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-aa9337f9-3b0e-4c3d-bb92-fd5aec6c3f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:29.345593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:29.413771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:29.713927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d3908-49c1-49ba-82e8-cbeb66b70591 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:29.716261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-fc4d3908-49c1-49ba-82e8-cbeb66b70591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:29.819952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d3908-49c1-49ba-82e8-cbeb66b70591 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:29.848559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:29.895575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cae42e-68bf-4ef8-b111-a14c1e784c81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:29.896225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cae42e-68bf-4ef8-b111-a14c1e784c81 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:29.896504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cae42e-68bf-4ef8-b111-a14c1e784c81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:29.896777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cae42e-68bf-4ef8-b111-a14c1e784c81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:29.898642 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c8cae42e-68bf-4ef8-b111-a14c1e784c81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:30.009482 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cae42e-68bf-4ef8-b111-a14c1e784c81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:30.010964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cae42e-68bf-4ef8-b111-a14c1e784c81 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:30.030742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 731/1458] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:32.087223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd521d9-e1c6-4f00-a4f4-1357cd3e54de None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:32.087737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd521d9-e1c6-4f00-a4f4-1357cd3e54de None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:32.087737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd521d9-e1c6-4f00-a4f4-1357cd3e54de None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:32.089067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:32.095514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6bb61-464e-47ff-adb8-61ff050a942b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:32.095948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6bb61-464e-47ff-adb8-61ff050a942b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:32.096093 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6bb61-464e-47ff-adb8-61ff050a942b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:32.164057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e1e6bb61-464e-47ff-adb8-61ff050a942b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:32.165261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e1e6bb61-464e-47ff-adb8-61ff050a942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:32.174553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:32.263337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:32.551736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd521d9-e1c6-4f00-a4f4-1357cd3e54de None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:32.553345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4bd521d9-e1c6-4f00-a4f4-1357cd3e54de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:32.660991 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd521d9-e1c6-4f00-a4f4-1357cd3e54de None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:32.720555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:32.740020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61084b8-0fc0-4479-ad2a-f539555342e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:32.740611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61084b8-0fc0-4479-ad2a-f539555342e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:32.740891 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61084b8-0fc0-4479-ad2a-f539555342e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:32.741165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61084b8-0fc0-4479-ad2a-f539555342e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:32.742775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c61084b8-0fc0-4479-ad2a-f539555342e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:32.860105 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61084b8-0fc0-4479-ad2a-f539555342e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:32.861466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61084b8-0fc0-4479-ad2a-f539555342e4 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:32.880010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 729/1461] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:34.859539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc95abe-6edd-4d59-88b3-268d0a5aacc1 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.860122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc95abe-6edd-4d59-88b3-268d0a5aacc1 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.860122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc95abe-6edd-4d59-88b3-268d0a5aacc1 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.861466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:34.866667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259f9c3-195e-4f09-8f46-7b211b5d3d62 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.867079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259f9c3-195e-4f09-8f46-7b211b5d3d62 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.867215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259f9c3-195e-4f09-8f46-7b211b5d3d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.934050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-4259f9c3-195e-4f09-8f46-7b211b5d3d62 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:34.935260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4259f9c3-195e-4f09-8f46-7b211b5d3d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:34.976808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:35.035688 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:35.319683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc95abe-6edd-4d59-88b3-268d0a5aacc1 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.321263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2cc95abe-6edd-4d59-88b3-268d0a5aacc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:35.428327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc95abe-6edd-4d59-88b3-268d0a5aacc1 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.467035 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:35.504657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1528489-3053-488f-b8f4-7ffdfbbcad4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.505270 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1528489-3053-488f-b8f4-7ffdfbbcad4d service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.505548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1528489-3053-488f-b8f4-7ffdfbbcad4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.505819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1528489-3053-488f-b8f4-7ffdfbbcad4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.507434 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c1528489-3053-488f-b8f4-7ffdfbbcad4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:35.618425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1528489-3053-488f-b8f4-7ffdfbbcad4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.619786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1528489-3053-488f-b8f4-7ffdfbbcad4d service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.639812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 734/1464] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:37.594655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70fdb5-6573-472c-893d-d0e7329ad718 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.595225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70fdb5-6573-472c-893d-d0e7329ad718 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.595225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70fdb5-6573-472c-893d-d0e7329ad718 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.596572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:37.601775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d60c4-53cb-4405-ac2f-ad0ae8a93bde None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.602214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d60c4-53cb-4405-ac2f-ad0ae8a93bde None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.602353 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d60c4-53cb-4405-ac2f-ad0ae8a93bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.670524 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-895d60c4-53cb-4405-ac2f-ad0ae8a93bde None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:37.671747 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-895d60c4-53cb-4405-ac2f-ad0ae8a93bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:37.692118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:37.778680 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:38.075296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70fdb5-6573-472c-893d-d0e7329ad718 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:38.076943 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-fc70fdb5-6573-472c-893d-d0e7329ad718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:38.181996 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70fdb5-6573-472c-893d-d0e7329ad718 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:38.249390 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:38.256533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bf8f5-db7e-421c-9bea-d745b5763891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:38.257140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bf8f5-db7e-421c-9bea-d745b5763891 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.257417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bf8f5-db7e-421c-9bea-d745b5763891 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.257687 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bf8f5-db7e-421c-9bea-d745b5763891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.259276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-546bf8f5-db7e-421c-9bea-d745b5763891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:38.368346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bf8f5-db7e-421c-9bea-d745b5763891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:38.369778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bf8f5-db7e-421c-9bea-d745b5763891 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:38.388265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 732/1467] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:40.369292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c8a3d-1970-46cd-b669-17c86abeb784 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:40.369826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c8a3d-1970-46cd-b669-17c86abeb784 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:40.369826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c8a3d-1970-46cd-b669-17c86abeb784 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:40.372197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:10:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:40.377578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f980b-a381-45c0-9c26-921fbff2e233 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:40.378037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f980b-a381-45c0-9c26-921fbff2e233 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:40.378146 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f980b-a381-45c0-9c26-921fbff2e233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:40.444991 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-266f980b-a381-45c0-9c26-921fbff2e233 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:40.446368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-266f980b-a381-45c0-9c26-921fbff2e233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:40.501234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:40.550563 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:40.842032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c8a3d-1970-46cd-b669-17c86abeb784 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:40.843586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b31c8a3d-1970-46cd-b669-17c86abeb784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:40.957671 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c8a3d-1970-46cd-b669-17c86abeb784 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:41.006762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:41.033674 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb715518-3ebb-4596-88e3-99344d37f189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:41.034290 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb715518-3ebb-4596-88e3-99344d37f189 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:41.034568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb715518-3ebb-4596-88e3-99344d37f189 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:41.034838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb715518-3ebb-4596-88e3-99344d37f189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:41.036473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-bb715518-3ebb-4596-88e3-99344d37f189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:41.149961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb715518-3ebb-4596-88e3-99344d37f189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:41.151484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb715518-3ebb-4596-88e3-99344d37f189 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:41.172680 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 737/1470] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:50.548663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=22 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:51.167842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=66 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:01.235747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77210f78-9752-4f45-a233-89783ddcde24 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:01.236342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77210f78-9752-4f45-a233-89783ddcde24 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:01.236342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77210f78-9752-4f45-a233-89783ddcde24 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:01.237690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 734/1471] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:11:01.242984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32928a-e973-4794-8da1-1f1123f1e13b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:01.243417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32928a-e973-4794-8da1-1f1123f1e13b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:01.243528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32928a-e973-4794-8da1-1f1123f1e13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:01.313791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-aa32928a-e973-4794-8da1-1f1123f1e13b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:01.315019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-aa32928a-e973-4794-8da1-1f1123f1e13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:01.418750 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 738/1472] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:01.722869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68140aa6-8e70-4973-bf54-4b4c1d87a879 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:01.723148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68140aa6-8e70-4973-bf54-4b4c1d87a879 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:01.723420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68140aa6-8e70-4973-bf54-4b4c1d87a879 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:01.724953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 735/1473] 10.222.0.44 () {54 vars in 967 bytes} [Tue Aug 30 01:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:11:01.730157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5b13b-5dbf-467d-a2e0-222f66117e7f None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:01.730443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5b13b-5dbf-467d-a2e0-222f66117e7f None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:01.730821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5b13b-5dbf-467d-a2e0-222f66117e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:01.809428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-bff5b13b-5dbf-467d-a2e0-222f66117e7f None None] MFA Rules not processed for user `92c9d444b3294ed18c3d488cf0a3eeb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:01.810618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-bff5b13b-5dbf-467d-a2e0-222f66117e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:01.831403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:01.887143 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 739/1474] 10.222.0.44 () {58 vars in 1064 bytes} [Tue Aug 30 01:11:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:01.894807 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68140aa6-8e70-4973-bf54-4b4c1d87a879 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:01.896414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-68140aa6-8e70-4973-bf54-4b4c1d87a879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:02.120556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68140aa6-8e70-4973-bf54-4b4c1d87a879 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:02.194784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd649e2-8cc2-4ea7-8562-9c1e80fdf87c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c9d444b3294ed18c3d488cf0a3eeb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:02.195384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd649e2-8cc2-4ea7-8562-9c1e80fdf87c service glance] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:02.195654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd649e2-8cc2-4ea7-8562-9c1e80fdf87c service glance] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:02.195918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd649e2-8cc2-4ea7-8562-9c1e80fdf87c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:02.197517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ccd649e2-8cc2-4ea7-8562-9c1e80fdf87c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:02.252074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:02.306301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd649e2-8cc2-4ea7-8562-9c1e80fdf87c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:02.307736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd649e2-8cc2-4ea7-8562-9c1e80fdf87c service glance] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:02.325902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 736/1475] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:04.346230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596769b-035c-4961-a6ef-06b3dd2d76b4 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:04.346832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596769b-035c-4961-a6ef-06b3dd2d76b4 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:04.346832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596769b-035c-4961-a6ef-06b3dd2d76b4 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:04.348164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 740/1476] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:11:04.353513 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e65522-3ea6-4b5b-b6a3-fcffc599ed01 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:04.353976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e65522-3ea6-4b5b-b6a3-fcffc599ed01 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:04.354084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e65522-3ea6-4b5b-b6a3-fcffc599ed01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:04.419475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-49e65522-3ea6-4b5b-b6a3-fcffc599ed01 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:04.420694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-49e65522-3ea6-4b5b-b6a3-fcffc599ed01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:04.518049 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 737/1477] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:04.795555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596769b-035c-4961-a6ef-06b3dd2d76b4 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:04.797159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f596769b-035c-4961-a6ef-06b3dd2d76b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:04.926287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596769b-035c-4961-a6ef-06b3dd2d76b4 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:04.976227 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:05.022743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a9a08-e2cd-4825-ba21-b303a8b626c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:05.023347 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a9a08-e2cd-4825-ba21-b303a8b626c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:05.023618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a9a08-e2cd-4825-ba21-b303a8b626c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:05.023881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a9a08-e2cd-4825-ba21-b303a8b626c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:05.024825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a9a08-e2cd-4825-ba21-b303a8b626c5 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:05.026585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: WARNING py.warnings [None req-054a9a08-e2cd-4825-ba21-b303a8b626c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:11:05.026585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: warnings.warn(msg) Aug 30 01:11:05.026585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: Aug 30 01:11:05.027781 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a9a08-e2cd-4825-ba21-b303a8b626c5 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:05.047022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 741/1478] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Aug 30 01:11:04 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:07.005566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94874ca4-1ccf-4156-8a49-2b3ad3f41cca None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:07.006128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94874ca4-1ccf-4156-8a49-2b3ad3f41cca None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:07.006128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94874ca4-1ccf-4156-8a49-2b3ad3f41cca None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:07.007452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 738/1479] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:11:07.013245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e872b2b-3839-44d8-8ca8-6fddb05511ca None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:07.013724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e872b2b-3839-44d8-8ca8-6fddb05511ca None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:07.013829 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e872b2b-3839-44d8-8ca8-6fddb05511ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:07.082199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-3e872b2b-3839-44d8-8ca8-6fddb05511ca None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:07.083400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3e872b2b-3839-44d8-8ca8-6fddb05511ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:07.314611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 742/1480] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:07.606026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94874ca4-1ccf-4156-8a49-2b3ad3f41cca None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:07.607425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-94874ca4-1ccf-4156-8a49-2b3ad3f41cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:07.617863 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:07.758319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94874ca4-1ccf-4156-8a49-2b3ad3f41cca None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:07.837973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:07.872252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ebb64-3f66-4d37-85ee-00d92a8f52b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:07.872836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ebb64-3f66-4d37-85ee-00d92a8f52b8 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:07.873036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ebb64-3f66-4d37-85ee-00d92a8f52b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:07.873304 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ebb64-3f66-4d37-85ee-00d92a8f52b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:07.874936 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7f6ebb64-3f66-4d37-85ee-00d92a8f52b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:07.967436 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ebb64-3f66-4d37-85ee-00d92a8f52b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:07.969079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ebb64-3f66-4d37-85ee-00d92a8f52b8 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:07.987678 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 739/1481] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:10.123677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da087dfa-a69f-4253-83ef-1148ac4230c4 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:10.124182 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da087dfa-a69f-4253-83ef-1148ac4230c4 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:10.124451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da087dfa-a69f-4253-83ef-1148ac4230c4 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:10.125922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 743/1482] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:11:10.131705 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65426bcd-2e11-4378-a800-98c2df241bef None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:10.132248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65426bcd-2e11-4378-a800-98c2df241bef None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:10.132346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65426bcd-2e11-4378-a800-98c2df241bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:10.203307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-65426bcd-2e11-4378-a800-98c2df241bef None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:10.204631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-65426bcd-2e11-4378-a800-98c2df241bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:10.255760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:10.307264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 740/1483] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:10.621675 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da087dfa-a69f-4253-83ef-1148ac4230c4 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:10.623183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-da087dfa-a69f-4253-83ef-1148ac4230c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:10.647124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:10.737138 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da087dfa-a69f-4253-83ef-1148ac4230c4 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:10.813829 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1187ac2-2aa8-4c1e-88fe-92c0dce57eb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:10.814433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1187ac2-2aa8-4c1e-88fe-92c0dce57eb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:10.814711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1187ac2-2aa8-4c1e-88fe-92c0dce57eb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:10.814982 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1187ac2-2aa8-4c1e-88fe-92c0dce57eb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:10.816769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c1187ac2-2aa8-4c1e-88fe-92c0dce57eb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:10.897009 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:10.935165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1187ac2-2aa8-4c1e-88fe-92c0dce57eb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:10.936651 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1187ac2-2aa8-4c1e-88fe-92c0dce57eb6 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:10.955492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 744/1484] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:12.940831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7b8db-9926-439f-9b48-5a145cbcc217 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:12.941400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7b8db-9926-439f-9b48-5a145cbcc217 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:12.941400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7b8db-9926-439f-9b48-5a145cbcc217 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:12.942858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 741/1485] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:11:12.948812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e56a1-393a-471b-9004-59d52a728f55 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:12.949636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e56a1-393a-471b-9004-59d52a728f55 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:12.949636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e56a1-393a-471b-9004-59d52a728f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:13.018111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-993e56a1-393a-471b-9004-59d52a728f55 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:13.019232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-993e56a1-393a-471b-9004-59d52a728f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:13.117998 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 745/1486] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:13.414563 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7b8db-9926-439f-9b48-5a145cbcc217 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:13.416118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a4a7b8db-9926-439f-9b48-5a145cbcc217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:13.523641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7b8db-9926-439f-9b48-5a145cbcc217 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:13.597489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:13.599450 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394e786-3182-430d-8f83-10afa2076041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:13.600098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394e786-3182-430d-8f83-10afa2076041 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:13.600563 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394e786-3182-430d-8f83-10afa2076041 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:13.600889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394e786-3182-430d-8f83-10afa2076041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:13.603045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9394e786-3182-430d-8f83-10afa2076041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:13.714536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394e786-3182-430d-8f83-10afa2076041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:13.715941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394e786-3182-430d-8f83-10afa2076041 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:13.733973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 742/1487] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:15.753929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef7317-9b71-4815-a637-a70d05b9f767 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:15.754467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef7317-9b71-4815-a637-a70d05b9f767 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:15.754587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef7317-9b71-4815-a637-a70d05b9f767 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:15.755983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 746/1488] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:11:15.762008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46083b-4144-4779-aa5f-e2f92d338519 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:15.762469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46083b-4144-4779-aa5f-e2f92d338519 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:15.762578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46083b-4144-4779-aa5f-e2f92d338519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:15.832325 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-0c46083b-4144-4779-aa5f-e2f92d338519 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:15.833547 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0c46083b-4144-4779-aa5f-e2f92d338519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:15.886224 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:15.934550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 743/1489] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:16.238671 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef7317-9b71-4815-a637-a70d05b9f767 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:16.240229 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4bef7317-9b71-4815-a637-a70d05b9f767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:16.276296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:16.347923 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef7317-9b71-4815-a637-a70d05b9f767 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:16.425676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e2161-da87-4ad1-ad66-4db03bf3b066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:16.426284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e2161-da87-4ad1-ad66-4db03bf3b066 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:16.426558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e2161-da87-4ad1-ad66-4db03bf3b066 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:16.426823 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e2161-da87-4ad1-ad66-4db03bf3b066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:16.428466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9c0e2161-da87-4ad1-ad66-4db03bf3b066 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:16.513161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:16.544409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e2161-da87-4ad1-ad66-4db03bf3b066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:16.545899 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e2161-da87-4ad1-ad66-4db03bf3b066 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:16.566690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 747/1490] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:18.852137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9e24a-e825-459d-971e-6915e3af1747 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:18.852790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9e24a-e825-459d-971e-6915e3af1747 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:18.852790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9e24a-e825-459d-971e-6915e3af1747 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:18.854016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 744/1491] 10.222.0.44 () {54 vars in 927 bytes} [Tue Aug 30 01:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:11:18.859523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5691a-b964-460c-bfd2-1c5bd583af55 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:18.860021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5691a-b964-460c-bfd2-1c5bd583af55 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:18.860135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5691a-b964-460c-bfd2-1c5bd583af55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:18.930756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7cf5691a-b964-460c-bfd2-1c5bd583af55 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:18.931984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7cf5691a-b964-460c-bfd2-1c5bd583af55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:19.046292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 748/1492] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Aug 30 01:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:19.339310 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9e24a-e825-459d-971e-6915e3af1747 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:19.340883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4fd9e24a-e825-459d-971e-6915e3af1747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:19.454137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9e24a-e825-459d-971e-6915e3af1747 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:19.528084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91d540-0ac0-48f7-87a8-b82318d3984e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:19.528686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91d540-0ac0-48f7-87a8-b82318d3984e service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:19.528964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91d540-0ac0-48f7-87a8-b82318d3984e service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:19.529235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91d540-0ac0-48f7-87a8-b82318d3984e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:19.530833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7a91d540-0ac0-48f7-87a8-b82318d3984e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:19.539985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:19.643469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91d540-0ac0-48f7-87a8-b82318d3984e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:19.644842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91d540-0ac0-48f7-87a8-b82318d3984e service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:19.664300 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 745/1493] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:29.044353 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=82 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:29.661900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=50 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:42.350463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759dffc-be91-4880-ad23-5e1f7c5729cb None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.351069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759dffc-be91-4880-ad23-5e1f7c5729cb None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.351069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759dffc-be91-4880-ad23-5e1f7c5729cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.432516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-c759dffc-be91-4880-ad23-5e1f7c5729cb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:42.433755 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c759dffc-be91-4880-ad23-5e1f7c5729cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:42.533773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 749/1494] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:42.542476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b34c82-a29e-484b-8b72-9546b68c5ed8 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.543040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b34c82-a29e-484b-8b72-9546b68c5ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.543040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b34c82-a29e-484b-8b72-9546b68c5ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.620010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-57b34c82-a29e-484b-8b72-9546b68c5ed8 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:42.621154 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-57b34c82-a29e-484b-8b72-9546b68c5ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:42.719101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 746/1495] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:12:42.723772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759dffc-be91-4880-ad23-5e1f7c5729cb None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:42.725006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c759dffc-be91-4880-ad23-5e1f7c5729cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:42.736893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759dffc-be91-4880-ad23-5e1f7c5729cb None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:42.797935 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:42.832456 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefc219-707d-4d2d-bba5-0a996005dcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:42.833063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefc219-707d-4d2d-bba5-0a996005dcde admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.833338 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefc219-707d-4d2d-bba5-0a996005dcde admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.833603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefc219-707d-4d2d-bba5-0a996005dcde admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.834246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefc219-707d-4d2d-bba5-0a996005dcde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:12:42.834541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefc219-707d-4d2d-bba5-0a996005dcde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:42.835558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: WARNING py.warnings [None req-8eefc219-707d-4d2d-bba5-0a996005dcde admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:12:42.835558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: warnings.warn(msg) Aug 30 01:12:42.835558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: Aug 30 01:12:42.836477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefc219-707d-4d2d-bba5-0a996005dcde admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:42.848944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 750/1496] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:12:42.858270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b34c82-a29e-484b-8b72-9546b68c5ed8 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:42.867766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b34c82-a29e-484b-8b72-9546b68c5ed8 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:42.930470 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:42.965401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481e82a-1fda-46f8-b563-5173fa6816cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:42.966010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481e82a-1fda-46f8-b563-5173fa6816cb admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.966288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481e82a-1fda-46f8-b563-5173fa6816cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.966559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481e82a-1fda-46f8-b563-5173fa6816cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.967192 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481e82a-1fda-46f8-b563-5173fa6816cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:42.968966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: WARNING py.warnings [None req-3481e82a-1fda-46f8-b563-5173fa6816cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:12:42.968966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: warnings.warn(msg) Aug 30 01:12:42.968966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: Aug 30 01:12:42.970165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481e82a-1fda-46f8-b563-5173fa6816cb admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:43.028133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 747/1497] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:42 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:43.032416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefc219-707d-4d2d-bba5-0a996005dcde admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:43.033945 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8eefc219-707d-4d2d-bba5-0a996005dcde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:43.067948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefc219-707d-4d2d-bba5-0a996005dcde admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:43.167886 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff968a-d09b-49c7-b3d5-d8a812a81794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:43.168522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff968a-d09b-49c7-b3d5-d8a812a81794 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.168803 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff968a-d09b-49c7-b3d5-d8a812a81794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.169069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff968a-d09b-49c7-b3d5-d8a812a81794 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.169702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff968a-d09b-49c7-b3d5-d8a812a81794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:43.170717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: WARNING py.warnings [None req-66ff968a-d09b-49c7-b3d5-d8a812a81794 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:12:43.170717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: warnings.warn(msg) Aug 30 01:12:43.170717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: Aug 30 01:12:43.171923 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff968a-d09b-49c7-b3d5-d8a812a81794 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:43.217779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 751/1498] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:43 2022] POST /identity/v3/users => generated 290 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:43.223041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481e82a-1fda-46f8-b563-5173fa6816cb admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:43.225817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3481e82a-1fda-46f8-b563-5173fa6816cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:43.296416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481e82a-1fda-46f8-b563-5173fa6816cb admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:43.383401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:43.399156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610d4e5-d57d-4e91-aa72-44146658218e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:43.400632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610d4e5-d57d-4e91-aa72-44146658218e admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.400911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610d4e5-d57d-4e91-aa72-44146658218e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.401179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610d4e5-d57d-4e91-aa72-44146658218e admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.401828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610d4e5-d57d-4e91-aa72-44146658218e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:43.402835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: WARNING py.warnings [None req-e610d4e5-d57d-4e91-aa72-44146658218e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:12:43.402835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: warnings.warn(msg) Aug 30 01:12:43.402835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: Aug 30 01:12:43.403720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610d4e5-d57d-4e91-aa72-44146658218e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:43.417228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 748/1499] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:12:43.421341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff968a-d09b-49c7-b3d5-d8a812a81794 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:43.422869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-66ff968a-d09b-49c7-b3d5-d8a812a81794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:43.434946 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff968a-d09b-49c7-b3d5-d8a812a81794 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:43.460901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:43.536103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5457a1a-9099-4274-ac45-a3f223ef7a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:43.536673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5457a1a-9099-4274-ac45-a3f223ef7a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.536954 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5457a1a-9099-4274-ac45-a3f223ef7a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.537218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5457a1a-9099-4274-ac45-a3f223ef7a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.537879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5457a1a-9099-4274-ac45-a3f223ef7a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:43.538891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: WARNING py.warnings [None req-d5457a1a-9099-4274-ac45-a3f223ef7a52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:12:43.538891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: warnings.warn(msg) Aug 30 01:12:43.538891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: Aug 30 01:12:43.539760 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5457a1a-9099-4274-ac45-a3f223ef7a52 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:43.554213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 752/1500] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:43.558667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610d4e5-d57d-4e91-aa72-44146658218e admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:43.560353 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e610d4e5-d57d-4e91-aa72-44146658218e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:43.597653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610d4e5-d57d-4e91-aa72-44146658218e admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:43.696956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38778e2-662d-4df5-853d-5ae3771a330f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:43.697572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38778e2-662d-4df5-853d-5ae3771a330f admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.697842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38778e2-662d-4df5-853d-5ae3771a330f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.698133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38778e2-662d-4df5-853d-5ae3771a330f admin admin] PATH_INFO: `/v3/projects/96e98d608aa641ccad083064032e3637/users/de7ac6b257944ca8b0ddb95e6679b9a9/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.723171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38778e2-662d-4df5-853d-5ae3771a330f admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e98d608aa641ccad083064032e3637, user_id=de7ac6b257944ca8b0ddb95e6679b9a9, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:43.724241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: WARNING py.warnings [None req-f38778e2-662d-4df5-853d-5ae3771a330f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:12:43.724241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: warnings.warn(msg) Aug 30 01:12:43.724241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: Aug 30 01:12:43.726179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38778e2-662d-4df5-853d-5ae3771a330f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:43.752710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 749/1501] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:43 2022] PUT /identity/v3/projects/96e98d608aa641ccad083064032e3637/users/de7ac6b257944ca8b0ddb95e6679b9a9/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:43.765157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32033eb4-4762-4caf-8e36-90da5b92d7de None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.765435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32033eb4-4762-4caf-8e36-90da5b92d7de None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.765702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32033eb4-4762-4caf-8e36-90da5b92d7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.824041 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-32033eb4-4762-4caf-8e36-90da5b92d7de None None] MFA Rules not processed for user `de7ac6b257944ca8b0ddb95e6679b9a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:43.825186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-32033eb4-4762-4caf-8e36-90da5b92d7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:43.905411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 753/1502] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:12:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:43.919486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec308-d094-46fe-aa6c-0636aef0aacd None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.919768 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec308-d094-46fe-aa6c-0636aef0aacd None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.920074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec308-d094-46fe-aa6c-0636aef0aacd None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.921415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 750/1503] 10.222.0.44 () {54 vars in 964 bytes} [Tue Aug 30 01:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:12:43.926888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffbd23-edf4-49b8-8370-c7ee594428a4 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.927176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffbd23-edf4-49b8-8370-c7ee594428a4 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.927473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffbd23-edf4-49b8-8370-c7ee594428a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.959236 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:44.006163 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-20ffbd23-edf4-49b8-8370-c7ee594428a4 None None] MFA Rules not processed for user `534da5b47dda43dabaab2cb6aa683342`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:44.007377 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-20ffbd23-edf4-49b8-8370-c7ee594428a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:44.084418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 754/1504] 10.222.0.44 () {58 vars in 1061 bytes} [Tue Aug 30 01:12:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:44.092843 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec308-d094-46fe-aa6c-0636aef0aacd None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:44.094386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-329ec308-d094-46fe-aa6c-0636aef0aacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:44.144430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:44.200973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec308-d094-46fe-aa6c-0636aef0aacd None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:44.278027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff673a-4007-4610-a9d9-66fb701ebc2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:44.278633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff673a-4007-4610-a9d9-66fb701ebc2c service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:44.278910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff673a-4007-4610-a9d9-66fb701ebc2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:44.279177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff673a-4007-4610-a9d9-66fb701ebc2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:44.280822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-21ff673a-4007-4610-a9d9-66fb701ebc2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:44.395003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff673a-4007-4610-a9d9-66fb701ebc2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:44.396545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff673a-4007-4610-a9d9-66fb701ebc2c service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:44.400538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:44.416108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 751/1505] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:12:44.461574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953bdfe-2243-4798-92c5-7e84ccd5a201 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:44.462113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953bdfe-2243-4798-92c5-7e84ccd5a201 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:44.462225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953bdfe-2243-4798-92c5-7e84ccd5a201 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:44.463506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 755/1506] 10.222.0.44 () {52 vars in 830 bytes} [Tue Aug 30 01:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:12:44.467282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff673a-4007-4610-a9d9-66fb701ebc2c service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:44.468530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-21ff673a-4007-4610-a9d9-66fb701ebc2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:44.569709 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff673a-4007-4610-a9d9-66fb701ebc2c service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:44.664675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:44.666391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:44.666668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:44.666938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] PATH_INFO: `/v3/users/de7ac6b257944ca8b0ddb95e6679b9a9` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:44.679121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7ac6b257944ca8b0ddb95e6679b9a9)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:44.680174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: WARNING py.warnings [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:12:44.680174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: warnings.warn(msg) Aug 30 01:12:44.680174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: Aug 30 01:12:44.682608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:44.692942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:44.757083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7ac6b257944ca8b0ddb95e6679b9a9'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:44.768018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7ac6b257944ca8b0ddb95e6679b9a9'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:44.768542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7ac6b257944ca8b0ddb95e6679b9a9'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:44.768964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7ac6b257944ca8b0ddb95e6679b9a9'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:44.793359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7ac6b257944ca8b0ddb95e6679b9a9'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:44.812532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 752/1507] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:44 2022] DELETE /identity/v3/users/de7ac6b257944ca8b0ddb95e6679b9a9 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:44.816987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca9182-b9b0-4463-8f32-dbeaec9c8410 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:44.817266 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca9182-b9b0-4463-8f32-dbeaec9c8410 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:44.817533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca9182-b9b0-4463-8f32-dbeaec9c8410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:44.875174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-c7ca9182-b9b0-4463-8f32-dbeaec9c8410 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:44.876462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c7ca9182-b9b0-4463-8f32-dbeaec9c8410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:44.953729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:44.967231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 756/1508] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:12:44.977327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:44.979231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:45.082301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c6fa5-52a1-4b89-aa3b-786c15403448 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:45.140222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:45.162806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a511c-c7fa-4ce3-abe2-202c9ff1607a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:45.163420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a511c-c7fa-4ce3-abe2-202c9ff1607a service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:45.163693 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a511c-c7fa-4ce3-abe2-202c9ff1607a service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:45.163958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a511c-c7fa-4ce3-abe2-202c9ff1607a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:45.166070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-df9a511c-c7fa-4ce3-abe2-202c9ff1607a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:45.280974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a511c-c7fa-4ce3-abe2-202c9ff1607a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:45.282321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a511c-c7fa-4ce3-abe2-202c9ff1607a service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:45.300284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 753/1509] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:12:45.664978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca9182-b9b0-4463-8f32-dbeaec9c8410 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:45.666458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c7ca9182-b9b0-4463-8f32-dbeaec9c8410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:45.678856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca9182-b9b0-4463-8f32-dbeaec9c8410 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:45.788316 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:45.789025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:45.789249 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:45.789517 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] PATH_INFO: `/v3/projects/96e98d608aa641ccad083064032e3637` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:45.802002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96e98d608aa641ccad083064032e3637)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:45.803047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: WARNING py.warnings [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:12:45.803047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: warnings.warn(msg) Aug 30 01:12:45.803047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: Aug 30 01:12:45.804333 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:45.865680 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96e98d608aa641ccad083064032e3637', 'request_id': 'req-e8058f3f-1bf5-4c96-89e8-4cf682177e31'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:45.876826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96e98d608aa641ccad083064032e3637', 'request_id': 'req-e8058f3f-1bf5-4c96-89e8-4cf682177e31'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:45.884288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96e98d608aa641ccad083064032e3637', 'request_id': 'req-e8058f3f-1bf5-4c96-89e8-4cf682177e31'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:45.917359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] The token cache is being invalidate because project 96e98d608aa641ccad083064032e3637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:45.917648 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:45.918083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e8058f3f-1bf5-4c96-89e8-4cf682177e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:45.919538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 757/1510] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:45 2022] DELETE /identity/v3/projects/96e98d608aa641ccad083064032e3637 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:55.298028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=62 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:55.915931 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:19.460554 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b870e86-4df5-4acb-b8d9-19a3609942a8 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:19.461247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b870e86-4df5-4acb-b8d9-19a3609942a8 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:19.461247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b870e86-4df5-4acb-b8d9-19a3609942a8 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:19.462598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:14:19.468065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31637dd-5ead-4afb-981b-c9fcdabe085c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:19.468621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31637dd-5ead-4afb-981b-c9fcdabe085c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:19.468725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31637dd-5ead-4afb-981b-c9fcdabe085c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:19.607753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e31637dd-5ead-4afb-981b-c9fcdabe085c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:19.609972 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e31637dd-5ead-4afb-981b-c9fcdabe085c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:19.761478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:14:20.038256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b870e86-4df5-4acb-b8d9-19a3609942a8 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:20.040755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8b870e86-4df5-4acb-b8d9-19a3609942a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:20.200454 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b870e86-4df5-4acb-b8d9-19a3609942a8 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:20.207699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2439cf-1404-4fcd-909c-d16fdbcb4551 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:20.208469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2439cf-1404-4fcd-909c-d16fdbcb4551 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:20.208745 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2439cf-1404-4fcd-909c-d16fdbcb4551 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:20.209036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2439cf-1404-4fcd-909c-d16fdbcb4551 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:20.224862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2439cf-1404-4fcd-909c-d16fdbcb4551 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:20.227874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2439cf-1404-4fcd-909c-d16fdbcb4551 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:20.243987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 755/1513] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:14:20.514032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1de34-d00c-4a66-a78d-61b8f7d857a5 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:20.514704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1de34-d00c-4a66-a78d-61b8f7d857a5 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:20.515048 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1de34-d00c-4a66-a78d-61b8f7d857a5 None None] PATH_INFO: `/v3` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:20.517309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 759/1514] 10.222.0.61 () {54 vars in 925 bytes} [Tue Aug 30 01:14:20 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:14:20.524714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da69e1-6c9a-44b3-8147-a3fcbfcc09b5 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:20.525675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da69e1-6c9a-44b3-8147-a3fcbfcc09b5 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:20.525675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da69e1-6c9a-44b3-8147-a3fcbfcc09b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:20.602238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-02da69e1-6c9a-44b3-8147-a3fcbfcc09b5 None None] MFA Rules not processed for user `08f34295f19b4d6ea405f1e8c8190db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:20.603410 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-02da69e1-6c9a-44b3-8147-a3fcbfcc09b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:20.720451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:20.728439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 756/1515] 10.222.0.61 () {58 vars in 1015 bytes} [Tue Aug 30 01:14:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:14:21.157514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1f829-4442-47d7-8c5d-09aad5e76fd9 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:21.157973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1f829-4442-47d7-8c5d-09aad5e76fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:21.158428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1f829-4442-47d7-8c5d-09aad5e76fd9 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:21.161519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 760/1516] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:14:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:14:21.170402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cc77a-c5ff-4250-b155-4c3d071e0fcb None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:21.170888 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cc77a-c5ff-4250-b155-4c3d071e0fcb None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:21.171374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cc77a-c5ff-4250-b155-4c3d071e0fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:21.204975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-957cc77a-c5ff-4250-b155-4c3d071e0fcb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:21.206500 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-957cc77a-c5ff-4250-b155-4c3d071e0fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:21.220446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 757/1517] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:21.497375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1f829-4442-47d7-8c5d-09aad5e76fd9 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:21.515505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1f829-4442-47d7-8c5d-09aad5e76fd9 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:21.523823 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4961fc8a-b1c1-411d-9a45-5e6297f38d81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:21.524744 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961fc8a-b1c1-411d-9a45-5e6297f38d81 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:21.525068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961fc8a-b1c1-411d-9a45-5e6297f38d81 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:21.525384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961fc8a-b1c1-411d-9a45-5e6297f38d81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:21.535861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4961fc8a-b1c1-411d-9a45-5e6297f38d81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:21.537686 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4961fc8a-b1c1-411d-9a45-5e6297f38d81 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:21.549131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 761/1518] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:14:30.935284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a7eb4-eecc-414b-aac7-d1e32e758d68 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:30.935920 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a7eb4-eecc-414b-aac7-d1e32e758d68 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:30.936270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a7eb4-eecc-414b-aac7-d1e32e758d68 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:30.938840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 758/1519] 10.222.0.44 () {54 vars in 961 bytes} [Tue Aug 30 01:14:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:14:30.945344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115be1f1-84e5-48d4-b9be-dd11f9cdea5a None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:30.945919 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115be1f1-84e5-48d4-b9be-dd11f9cdea5a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:30.945919 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115be1f1-84e5-48d4-b9be-dd11f9cdea5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:30.968150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-115be1f1-84e5-48d4-b9be-dd11f9cdea5a None None] MFA Rules not processed for user `08f34295f19b4d6ea405f1e8c8190db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:30.969411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-115be1f1-84e5-48d4-b9be-dd11f9cdea5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:30.979157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 762/1520] 10.222.0.44 () {58 vars in 1058 bytes} [Tue Aug 30 01:14:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:14:30.988118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a7eb4-eecc-414b-aac7-d1e32e758d68 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:31.013973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a7eb4-eecc-414b-aac7-d1e32e758d68 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:31.022140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c6594-1feb-4ae2-ad8a-8086aabdb555 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08f34295f19b4d6ea405f1e8c8190db5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:31.023073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c6594-1feb-4ae2-ad8a-8086aabdb555 service placement] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:31.023415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c6594-1feb-4ae2-ad8a-8086aabdb555 service placement] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:31.023732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c6594-1feb-4ae2-ad8a-8086aabdb555 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:31.033669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c6594-1feb-4ae2-ad8a-8086aabdb555 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:31.035780 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c6594-1feb-4ae2-ad8a-8086aabdb555 service placement] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:31.046772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 759/1521] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:14:32.858899 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7e03c-2df1-477d-bdc7-9e4f2fe59e7a None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:32.859490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7e03c-2df1-477d-bdc7-9e4f2fe59e7a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:32.859490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7e03c-2df1-477d-bdc7-9e4f2fe59e7a None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:32.860933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:14:32.870392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407229b-92eb-4e94-88fd-2ecafd74ebc9 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:32.871020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407229b-92eb-4e94-88fd-2ecafd74ebc9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:32.871329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407229b-92eb-4e94-88fd-2ecafd74ebc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:32.899247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-b407229b-92eb-4e94-88fd-2ecafd74ebc9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:32.900553 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b407229b-92eb-4e94-88fd-2ecafd74ebc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:32.908920 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:33.070524 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7e03c-2df1-477d-bdc7-9e4f2fe59e7a None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:33.097143 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7e03c-2df1-477d-bdc7-9e4f2fe59e7a None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:33.105307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547301dd-baa3-430b-9656-51b70092d4eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:33.106349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547301dd-baa3-430b-9656-51b70092d4eb service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:33.106830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547301dd-baa3-430b-9656-51b70092d4eb service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:33.107299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547301dd-baa3-430b-9656-51b70092d4eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:33.121440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547301dd-baa3-430b-9656-51b70092d4eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:33.123340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547301dd-baa3-430b-9656-51b70092d4eb service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:33.135501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 764/1524] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:34.040486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0274d8e-f151-4330-b132-282398de81c6 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:34.041231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0274d8e-f151-4330-b132-282398de81c6 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:34.041231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0274d8e-f151-4330-b132-282398de81c6 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:34.042386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:14:34.048460 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb0909-9271-4e05-a356-03760760503e None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:34.049065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb0909-9271-4e05-a356-03760760503e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:34.049065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb0909-9271-4e05-a356-03760760503e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:34.072420 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-c6eb0909-9271-4e05-a356-03760760503e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:34.073658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c6eb0909-9271-4e05-a356-03760760503e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:34.082040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:14:34.368950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0274d8e-f151-4330-b132-282398de81c6 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:34.389767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0274d8e-f151-4330-b132-282398de81c6 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:34.396975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bb690-b807-4130-9f6d-f6a3942d526a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:34.397941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bb690-b807-4130-9f6d-f6a3942d526a service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:34.398325 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bb690-b807-4130-9f6d-f6a3942d526a service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:34.398766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bb690-b807-4130-9f6d-f6a3942d526a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:34.408656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895bb690-b807-4130-9f6d-f6a3942d526a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:34.411096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895bb690-b807-4130-9f6d-f6a3942d526a service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:34.422068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 762/1527] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:38.258435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e02f2-65af-4f4a-b556-1708a7207875 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:38.259077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e02f2-65af-4f4a-b556-1708a7207875 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:38.259077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e02f2-65af-4f4a-b556-1708a7207875 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:38.260547 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:14:38.267171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb45ed-91a5-4acd-b749-258f2eb9b909 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:38.267859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb45ed-91a5-4acd-b749-258f2eb9b909 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:38.267859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb45ed-91a5-4acd-b749-258f2eb9b909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:38.290243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-24bb45ed-91a5-4acd-b749-258f2eb9b909 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:38.291492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-24bb45ed-91a5-4acd-b749-258f2eb9b909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:38.300966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:38.600249 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e02f2-65af-4f4a-b556-1708a7207875 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:38.602065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1f9e02f2-65af-4f4a-b556-1708a7207875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:38.639239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:38.693681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e02f2-65af-4f4a-b556-1708a7207875 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:38.697678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fdc9d-128c-4612-b206-3e666f41b800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:38.698369 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fdc9d-128c-4612-b206-3e666f41b800 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:38.698745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fdc9d-128c-4612-b206-3e666f41b800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:38.699115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fdc9d-128c-4612-b206-3e666f41b800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:38.714497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951fdc9d-128c-4612-b206-3e666f41b800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:38.717112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951fdc9d-128c-4612-b206-3e666f41b800 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:38.735821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 767/1530] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:40.802209 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ad53b-a973-4ff7-9271-2cc394cc9548 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:40.802897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ad53b-a973-4ff7-9271-2cc394cc9548 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:40.802897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ad53b-a973-4ff7-9271-2cc394cc9548 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:40.804488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:14:40.810579 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f609c-c3c8-40b2-a967-cb4d0871f0ae None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:40.811120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f609c-c3c8-40b2-a967-cb4d0871f0ae None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:40.811214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f609c-c3c8-40b2-a967-cb4d0871f0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:40.835598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-134f609c-c3c8-40b2-a967-cb4d0871f0ae None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:40.836859 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-134f609c-c3c8-40b2-a967-cb4d0871f0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:40.845114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:41.140537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ad53b-a973-4ff7-9271-2cc394cc9548 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:41.152974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ad53b-a973-4ff7-9271-2cc394cc9548 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:41.156935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70523784-9c92-42a5-8fab-7ea83a52002c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:41.157589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70523784-9c92-42a5-8fab-7ea83a52002c service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:41.157875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70523784-9c92-42a5-8fab-7ea83a52002c service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:41.158142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70523784-9c92-42a5-8fab-7ea83a52002c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:41.169470 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70523784-9c92-42a5-8fab-7ea83a52002c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:41.170970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70523784-9c92-42a5-8fab-7ea83a52002c service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:41.183450 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 765/1533] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:44.932468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc6dd7-cb39-4c00-b6f2-ee5ee57a7320 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:44.933083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc6dd7-cb39-4c00-b6f2-ee5ee57a7320 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:44.933223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc6dd7-cb39-4c00-b6f2-ee5ee57a7320 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:44.934759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:14:44.941521 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c109d-b560-4af9-bc36-ab9ff93d23f9 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:44.942225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c109d-b560-4af9-bc36-ab9ff93d23f9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:44.942225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c109d-b560-4af9-bc36-ab9ff93d23f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:44.966369 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-4e2c109d-b560-4af9-bc36-ab9ff93d23f9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:44.967516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4e2c109d-b560-4af9-bc36-ab9ff93d23f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:44.976200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:45.570289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc6dd7-cb39-4c00-b6f2-ee5ee57a7320 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:45.572245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e8fc6dd7-cb39-4c00-b6f2-ee5ee57a7320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:45.678535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc6dd7-cb39-4c00-b6f2-ee5ee57a7320 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:45.682314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8d756-e0e7-4884-a60d-f2d4e0ffae10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:45.682942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8d756-e0e7-4884-a60d-f2d4e0ffae10 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:45.683219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8d756-e0e7-4884-a60d-f2d4e0ffae10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:45.683497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8d756-e0e7-4884-a60d-f2d4e0ffae10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:45.695246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8d756-e0e7-4884-a60d-f2d4e0ffae10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:45.706682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8d756-e0e7-4884-a60d-f2d4e0ffae10 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:45.718487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 770/1536] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:47.904140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2993e-c142-44fa-a57c-dea675a3b5ac None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:47.904800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2993e-c142-44fa-a57c-dea675a3b5ac None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:47.904800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2993e-c142-44fa-a57c-dea675a3b5ac None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:47.906095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:14:47.912918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24837fe7-238e-4732-a6e5-a1e979dbb910 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:47.913799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24837fe7-238e-4732-a6e5-a1e979dbb910 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:47.913799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24837fe7-238e-4732-a6e5-a1e979dbb910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:47.942821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-24837fe7-238e-4732-a6e5-a1e979dbb910 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:47.945021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-24837fe7-238e-4732-a6e5-a1e979dbb910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:47.959952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:48.575106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2993e-c142-44fa-a57c-dea675a3b5ac None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:48.577108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-aad2993e-c142-44fa-a57c-dea675a3b5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:48.593420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2993e-c142-44fa-a57c-dea675a3b5ac None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:48.598198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56808b15-dd64-4ae1-8f36-8d8d126e9ec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:48.598855 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56808b15-dd64-4ae1-8f36-8d8d126e9ec2 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:48.599148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56808b15-dd64-4ae1-8f36-8d8d126e9ec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:48.599456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56808b15-dd64-4ae1-8f36-8d8d126e9ec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:48.610343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56808b15-dd64-4ae1-8f36-8d8d126e9ec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:48.612742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56808b15-dd64-4ae1-8f36-8d8d126e9ec2 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:48.623850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 768/1539] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:57.934636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=63 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:58.618813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=37 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:17.881927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c812bc-fbbc-478c-adc8-881eb50601cc None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:17.883471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c812bc-fbbc-478c-adc8-881eb50601cc None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:17.883471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c812bc-fbbc-478c-adc8-881eb50601cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:17.929778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d0c812bc-fbbc-478c-adc8-881eb50601cc None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:17.931213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d0c812bc-fbbc-478c-adc8-881eb50601cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:17.947296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 772/1540] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:17.957467 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90480c27-eec5-40b6-b2c4-d26ce3ab8268 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:17.958072 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90480c27-eec5-40b6-b2c4-d26ce3ab8268 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:17.958072 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90480c27-eec5-40b6-b2c4-d26ce3ab8268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:17.968596 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176a4db-993a-4627-b253-52dc04bdad12 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:17.968808 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176a4db-993a-4627-b253-52dc04bdad12 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:17.969064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176a4db-993a-4627-b253-52dc04bdad12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:17.993629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-90480c27-eec5-40b6-b2c4-d26ce3ab8268 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:17.994663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-90480c27-eec5-40b6-b2c4-d26ce3ab8268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.019555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 769/1541] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.022760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47e088-a9fc-48c1-86bd-95012adabf39 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.023200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47e088-a9fc-48c1-86bd-95012adabf39 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.023635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47e088-a9fc-48c1-86bd-95012adabf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.030730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6176a4db-993a-4627-b253-52dc04bdad12 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.032607 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6176a4db-993a-4627-b253-52dc04bdad12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.048447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 773/1542] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:18.049328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176a4db-993a-4627-b253-52dc04bdad12 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.069422 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176a4db-993a-4627-b253-52dc04bdad12 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.083170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce222a-39ac-4da2-92a3-a78b931aa70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.084250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce222a-39ac-4da2-92a3-a78b931aa70b admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.084533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce222a-39ac-4da2-92a3-a78b931aa70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.084804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce222a-39ac-4da2-92a3-a78b931aa70b admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.085478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce222a-39ac-4da2-92a3-a78b931aa70b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:18.085783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce222a-39ac-4da2-92a3-a78b931aa70b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.087720 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce222a-39ac-4da2-92a3-a78b931aa70b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.090865 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-4f47e088-a9fc-48c1-86bd-95012adabf39 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.092096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4f47e088-a9fc-48c1-86bd-95012adabf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.111943 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 770/1543] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.113496 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5704a5-7618-423b-bae6-b23c01cdf021 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.113774 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5704a5-7618-423b-bae6-b23c01cdf021 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.114044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5704a5-7618-423b-bae6-b23c01cdf021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.114985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 774/1544] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:18.125106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca73513-f0f4-42a2-9b51-fc0cd06ee576 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.125636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca73513-f0f4-42a2-9b51-fc0cd06ee576 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.125918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca73513-f0f4-42a2-9b51-fc0cd06ee576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.144824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-3c5704a5-7618-423b-bae6-b23c01cdf021 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.146045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3c5704a5-7618-423b-bae6-b23c01cdf021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.166083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 771/1545] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.169813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5704a5-7618-423b-bae6-b23c01cdf021 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.169971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-0ca73513-f0f4-42a2-9b51-fc0cd06ee576 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.169971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0ca73513-f0f4-42a2-9b51-fc0cd06ee576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.178223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5704a5-7618-423b-bae6-b23c01cdf021 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.183578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0acaa7-2b8f-4dee-ba59-fe6adf80e9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.184442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0acaa7-2b8f-4dee-ba59-fe6adf80e9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.184715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0acaa7-2b8f-4dee-ba59-fe6adf80e9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.184977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0acaa7-2b8f-4dee-ba59-fe6adf80e9c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.185617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0acaa7-2b8f-4dee-ba59-fe6adf80e9c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.186594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 775/1546] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:18.187585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0acaa7-2b8f-4dee-ba59-fe6adf80e9c4 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.188631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d916f2a-f37c-4c6d-9d6a-4110d5d781d8 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.188913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d916f2a-f37c-4c6d-9d6a-4110d5d781d8 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.189182 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d916f2a-f37c-4c6d-9d6a-4110d5d781d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.216282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-2d916f2a-f37c-4c6d-9d6a-4110d5d781d8 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.217492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2d916f2a-f37c-4c6d-9d6a-4110d5d781d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.226765 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 776/1547] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.228479 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d916f2a-f37c-4c6d-9d6a-4110d5d781d8 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.243185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d916f2a-f37c-4c6d-9d6a-4110d5d781d8 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.243421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 772/1548] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/projects => generated 393 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:18.244782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc55e24-ace2-4a07-8dfb-ab4f57c9062c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.246600 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc55e24-ace2-4a07-8dfb-ab4f57c9062c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.246600 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc55e24-ace2-4a07-8dfb-ab4f57c9062c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.274220 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-7cc55e24-ace2-4a07-8dfb-ab4f57c9062c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.275829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7cc55e24-ace2-4a07-8dfb-ab4f57c9062c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.307451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29439e7a-ce67-4ac1-95e1-f5015412a283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.308507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29439e7a-ce67-4ac1-95e1-f5015412a283 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.309030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29439e7a-ce67-4ac1-95e1-f5015412a283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.309558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29439e7a-ce67-4ac1-95e1-f5015412a283 admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.310560 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29439e7a-ce67-4ac1-95e1-f5015412a283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:18.310967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29439e7a-ce67-4ac1-95e1-f5015412a283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.313314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29439e7a-ce67-4ac1-95e1-f5015412a283 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.325280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 777/1549] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:18.326540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7b2c2-6a41-47a0-9bf5-053dd3e6197e None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.326865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7b2c2-6a41-47a0-9bf5-053dd3e6197e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.327108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7b2c2-6a41-47a0-9bf5-053dd3e6197e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.346272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 773/1550] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.347691 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc55e24-ace2-4a07-8dfb-ab4f57c9062c None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.356300 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-94f7b2c2-6a41-47a0-9bf5-053dd3e6197e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.357730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-94f7b2c2-6a41-47a0-9bf5-053dd3e6197e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.358048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc55e24-ace2-4a07-8dfb-ab4f57c9062c None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.362822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d5ca7-d8cd-49d4-96fb-9a5e6b797748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.363420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d5ca7-d8cd-49d4-96fb-9a5e6b797748 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.363695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d5ca7-d8cd-49d4-96fb-9a5e6b797748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.363966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d5ca7-d8cd-49d4-96fb-9a5e6b797748 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.365180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d5ca7-d8cd-49d4-96fb-9a5e6b797748 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:18.365180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d5ca7-d8cd-49d4-96fb-9a5e6b797748 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.365909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: WARNING py.warnings [None req-3d8d5ca7-d8cd-49d4-96fb-9a5e6b797748 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:16:18.365909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: warnings.warn(msg) Aug 30 01:16:18.365909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: Aug 30 01:16:18.366975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d5ca7-d8cd-49d4-96fb-9a5e6b797748 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.367081 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 778/1551] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.368068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040f5d-9974-4ebe-acda-13f4e94bc3d5 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.368362 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040f5d-9974-4ebe-acda-13f4e94bc3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.368633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040f5d-9974-4ebe-acda-13f4e94bc3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.379400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 774/1552] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:18.380981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac41352-8da8-4e80-bdb0-c03181e3e4d4 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.381404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac41352-8da8-4e80-bdb0-c03181e3e4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.381836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac41352-8da8-4e80-bdb0-c03181e3e4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.396070 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-09040f5d-9974-4ebe-acda-13f4e94bc3d5 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.397299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-09040f5d-9974-4ebe-acda-13f4e94bc3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.403205 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-bac41352-8da8-4e80-bdb0-c03181e3e4d4 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.404428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-bac41352-8da8-4e80-bdb0-c03181e3e4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.406010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 779/1553] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.407243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b21e3-0cef-4528-adb5-fd6ec0e11032 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.407522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b21e3-0cef-4528-adb5-fd6ec0e11032 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.407799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b21e3-0cef-4528-adb5-fd6ec0e11032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.415066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 775/1554] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.416140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac41352-8da8-4e80-bdb0-c03181e3e4d4 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.428459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac41352-8da8-4e80-bdb0-c03181e3e4d4 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.430609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-021b21e3-0cef-4528-adb5-fd6ec0e11032 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.431810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-021b21e3-0cef-4528-adb5-fd6ec0e11032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.433555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72f841-cda6-45aa-a625-d7c409600b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.434240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72f841-cda6-45aa-a625-d7c409600b72 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.434517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72f841-cda6-45aa-a625-d7c409600b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.434785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72f841-cda6-45aa-a625-d7c409600b72 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.435422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72f841-cda6-45aa-a625-d7c409600b72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.436878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: WARNING py.warnings [None req-dd72f841-cda6-45aa-a625-d7c409600b72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:16:18.436878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: warnings.warn(msg) Aug 30 01:16:18.436878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: Aug 30 01:16:18.438683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72f841-cda6-45aa-a625-d7c409600b72 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.440763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 780/1555] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.441800 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b21e3-0cef-4528-adb5-fd6ec0e11032 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.451953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b21e3-0cef-4528-adb5-fd6ec0e11032 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.457421 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976506c0-24b4-433a-afa6-81de2b28d49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.458114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976506c0-24b4-433a-afa6-81de2b28d49b admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.458392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976506c0-24b4-433a-afa6-81de2b28d49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.458656 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976506c0-24b4-433a-afa6-81de2b28d49b admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.459293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976506c0-24b4-433a-afa6-81de2b28d49b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.460336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: WARNING py.warnings [None req-976506c0-24b4-433a-afa6-81de2b28d49b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:16:18.460336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: warnings.warn(msg) Aug 30 01:16:18.460336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: Aug 30 01:16:18.461521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976506c0-24b4-433a-afa6-81de2b28d49b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.464952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 776/1556] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/users => generated 297 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:18.466546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2b782-2aa6-4e87-bbb8-9931e78cfc9c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.466836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2b782-2aa6-4e87-bbb8-9931e78cfc9c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.467108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2b782-2aa6-4e87-bbb8-9931e78cfc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.490739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d3d2b782-2aa6-4e87-bbb8-9931e78cfc9c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.491946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d3d2b782-2aa6-4e87-bbb8-9931e78cfc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.498772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 781/1557] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:18.498772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6497c-ca1f-4bf6-9eab-9f093b844313 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.498772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6497c-ca1f-4bf6-9eab-9f093b844313 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.498772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6497c-ca1f-4bf6-9eab-9f093b844313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.519125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b7f6497c-ca1f-4bf6-9eab-9f093b844313 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.520481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b7f6497c-ca1f-4bf6-9eab-9f093b844313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.560767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 777/1558] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.561972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2b782-2aa6-4e87-bbb8-9931e78cfc9c None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.571877 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2b782-2aa6-4e87-bbb8-9931e78cfc9c None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.579684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31380659-6073-4e48-ac1b-d27170e48b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.580635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31380659-6073-4e48-ac1b-d27170e48b7b admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.580913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31380659-6073-4e48-ac1b-d27170e48b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.581183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31380659-6073-4e48-ac1b-d27170e48b7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.581853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31380659-6073-4e48-ac1b-d27170e48b7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.583811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31380659-6073-4e48-ac1b-d27170e48b7b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.587077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 782/1559] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.589180 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97281db0-d72d-45ca-8e79-feb9b608413a None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.589864 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97281db0-d72d-45ca-8e79-feb9b608413a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.590766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97281db0-d72d-45ca-8e79-feb9b608413a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.596898 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:18.615367 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-97281db0-d72d-45ca-8e79-feb9b608413a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.616767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-97281db0-d72d-45ca-8e79-feb9b608413a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.617324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 778/1560] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/projects => generated 355 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:18.618659 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfc1a0-38c7-407d-a3ad-d5c0ecf679af None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.618939 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfc1a0-38c7-407d-a3ad-d5c0ecf679af None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.619212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfc1a0-38c7-407d-a3ad-d5c0ecf679af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.627735 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:18.641891 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-e7dfc1a0-38c7-407d-a3ad-d5c0ecf679af None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:18.643403 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e7dfc1a0-38c7-407d-a3ad-d5c0ecf679af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:18.688506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 783/1561] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.688506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97281db0-d72d-45ca-8e79-feb9b608413a None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.700226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97281db0-d72d-45ca-8e79-feb9b608413a None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.705631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55ad9c-c260-462f-8c5c-53623e56ab1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.706286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55ad9c-c260-462f-8c5c-53623e56ab1d admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.706569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55ad9c-c260-462f-8c5c-53623e56ab1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.706838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55ad9c-c260-462f-8c5c-53623e56ab1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.707538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55ad9c-c260-462f-8c5c-53623e56ab1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:18.708225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55ad9c-c260-462f-8c5c-53623e56ab1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.709944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55ad9c-c260-462f-8c5c-53623e56ab1d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.724426 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 779/1562] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:18.725580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfc1a0-38c7-407d-a3ad-d5c0ecf679af None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.729642 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 784/1563] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:18.730829 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55ad9c-c260-462f-8c5c-53623e56ab1d admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.740470 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfc1a0-38c7-407d-a3ad-d5c0ecf679af None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.751514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55ad9c-c260-462f-8c5c-53623e56ab1d admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.751937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991cc09-5884-42d0-8b81-5f3f8ebee456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.752821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991cc09-5884-42d0-8b81-5f3f8ebee456 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.753250 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991cc09-5884-42d0-8b81-5f3f8ebee456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.753606 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991cc09-5884-42d0-8b81-5f3f8ebee456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.754350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991cc09-5884-42d0-8b81-5f3f8ebee456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.756331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991cc09-5884-42d0-8b81-5f3f8ebee456 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.756972 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdd773-fbed-456d-a7ff-c86c0fe03767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.757673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdd773-fbed-456d-a7ff-c86c0fe03767 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.758155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdd773-fbed-456d-a7ff-c86c0fe03767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.758429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdd773-fbed-456d-a7ff-c86c0fe03767 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.759440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdd773-fbed-456d-a7ff-c86c0fe03767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.761703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdd773-fbed-456d-a7ff-c86c0fe03767 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.772928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 780/1564] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:18.774608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991cc09-5884-42d0-8b81-5f3f8ebee456 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.788496 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991cc09-5884-42d0-8b81-5f3f8ebee456 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.794007 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 785/1565] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/users => generated 294 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:18.794007 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdd773-fbed-456d-a7ff-c86c0fe03767 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.797328 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dceed7b-87d4-4607-bdde-7660d42108ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.797992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dceed7b-87d4-4607-bdde-7660d42108ff admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.798221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dceed7b-87d4-4607-bdde-7660d42108ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.798490 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dceed7b-87d4-4607-bdde-7660d42108ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.799140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dceed7b-87d4-4607-bdde-7660d42108ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:18.799433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dceed7b-87d4-4607-bdde-7660d42108ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.801084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dceed7b-87d4-4607-bdde-7660d42108ff admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.803802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdd773-fbed-456d-a7ff-c86c0fe03767 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.808694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93bd3d-1f24-4194-a19d-4a644609c605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.809315 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93bd3d-1f24-4194-a19d-4a644609c605 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.809591 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93bd3d-1f24-4194-a19d-4a644609c605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.809866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93bd3d-1f24-4194-a19d-4a644609c605 admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.810516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93bd3d-1f24-4194-a19d-4a644609c605 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:18.810823 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93bd3d-1f24-4194-a19d-4a644609c605 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.812496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93bd3d-1f24-4194-a19d-4a644609c605 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.815227 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 781/1566] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:18.816108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dceed7b-87d4-4607-bdde-7660d42108ff admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.834658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 786/1567] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:18.834658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93bd3d-1f24-4194-a19d-4a644609c605 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.834999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dceed7b-87d4-4607-bdde-7660d42108ff admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.838258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93bd3d-1f24-4194-a19d-4a644609c605 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.839907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f5b1d-1b93-449c-b788-0a2ded783e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.840665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f5b1d-1b93-449c-b788-0a2ded783e1c admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.840944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f5b1d-1b93-449c-b788-0a2ded783e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.841210 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f5b1d-1b93-449c-b788-0a2ded783e1c admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.841855 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f5b1d-1b93-449c-b788-0a2ded783e1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.843708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea37ee0-f7cc-465b-89ca-dead9ecaf40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.843907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f5b1d-1b93-449c-b788-0a2ded783e1c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.844381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea37ee0-f7cc-465b-89ca-dead9ecaf40e admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.844662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea37ee0-f7cc-465b-89ca-dead9ecaf40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.844935 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea37ee0-f7cc-465b-89ca-dead9ecaf40e admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.845608 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea37ee0-f7cc-465b-89ca-dead9ecaf40e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:18.845921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea37ee0-f7cc-465b-89ca-dead9ecaf40e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.847761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea37ee0-f7cc-465b-89ca-dead9ecaf40e admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.861944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 787/1568] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:18.862759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea37ee0-f7cc-465b-89ca-dead9ecaf40e admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.874931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 782/1569] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/users => generated 278 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:18.874931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f5b1d-1b93-449c-b788-0a2ded783e1c admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.880472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea37ee0-f7cc-465b-89ca-dead9ecaf40e admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.883243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f5b1d-1b93-449c-b788-0a2ded783e1c admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.885257 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56003ee6-58d9-48c8-881a-8bb24513d20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.886416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56003ee6-58d9-48c8-881a-8bb24513d20c admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.886755 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56003ee6-58d9-48c8-881a-8bb24513d20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.887026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56003ee6-58d9-48c8-881a-8bb24513d20c admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.887694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56003ee6-58d9-48c8-881a-8bb24513d20c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:18.888016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56003ee6-58d9-48c8-881a-8bb24513d20c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.888738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1370bf6-9dbb-4cef-b414-ae2b1333e163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.889394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1370bf6-9dbb-4cef-b414-ae2b1333e163 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.889753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56003ee6-58d9-48c8-881a-8bb24513d20c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.889899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1370bf6-9dbb-4cef-b414-ae2b1333e163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.890172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1370bf6-9dbb-4cef-b414-ae2b1333e163 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.890856 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1370bf6-9dbb-4cef-b414-ae2b1333e163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.893112 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1370bf6-9dbb-4cef-b414-ae2b1333e163 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.903946 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 788/1570] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:18.906332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56003ee6-58d9-48c8-881a-8bb24513d20c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.919494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56003ee6-58d9-48c8-881a-8bb24513d20c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.924402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10e8ec-832c-4bba-a856-efc7e1fa7bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:18.924656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 783/1571] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:18.925636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1370bf6-9dbb-4cef-b414-ae2b1333e163 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.928345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10e8ec-832c-4bba-a856-efc7e1fa7bea admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:18.928633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10e8ec-832c-4bba-a856-efc7e1fa7bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:18.928907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10e8ec-832c-4bba-a856-efc7e1fa7bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:18.929589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10e8ec-832c-4bba-a856-efc7e1fa7bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:18.931255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10e8ec-832c-4bba-a856-efc7e1fa7bea admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:18.943325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 789/1572] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:18.944218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10e8ec-832c-4bba-a856-efc7e1fa7bea admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:18.954750 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10e8ec-832c-4bba-a856-efc7e1fa7bea admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:18.999929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1370bf6-9dbb-4cef-b414-ae2b1333e163 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.006151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15234d80-3182-4d76-9b87-935c9bc8a7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.006706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15234d80-3182-4d76-9b87-935c9bc8a7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.006985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15234d80-3182-4d76-9b87-935c9bc8a7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.007249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15234d80-3182-4d76-9b87-935c9bc8a7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.007907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15234d80-3182-4d76-9b87-935c9bc8a7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.009584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15234d80-3182-4d76-9b87-935c9bc8a7f7 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.015383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179d932-ae8b-4a4e-a305-7444fa73e552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.016560 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179d932-ae8b-4a4e-a305-7444fa73e552 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.016560 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179d932-ae8b-4a4e-a305-7444fa73e552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.016560 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179d932-ae8b-4a4e-a305-7444fa73e552 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.017225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179d932-ae8b-4a4e-a305-7444fa73e552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.019449 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179d932-ae8b-4a4e-a305-7444fa73e552 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.021781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 784/1573] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:19.022717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15234d80-3182-4d76-9b87-935c9bc8a7f7 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.034228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15234d80-3182-4d76-9b87-935c9bc8a7f7 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.038965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fdb3c-554a-4ab5-bf87-9b854f1cb25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.039574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fdb3c-554a-4ab5-bf87-9b854f1cb25a admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.039860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fdb3c-554a-4ab5-bf87-9b854f1cb25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.040318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fdb3c-554a-4ab5-bf87-9b854f1cb25a admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.040963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fdb3c-554a-4ab5-bf87-9b854f1cb25a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.043096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fdb3c-554a-4ab5-bf87-9b854f1cb25a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.059535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 790/1574] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:18 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:19.060236 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179d932-ae8b-4a4e-a305-7444fa73e552 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.083624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 785/1575] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:19 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:19.083624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fdb3c-554a-4ab5-bf87-9b854f1cb25a admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.083947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:19.092534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:19.138349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179d932-ae8b-4a4e-a305-7444fa73e552 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.161651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fdb3c-554a-4ab5-bf87-9b854f1cb25a admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.166646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035f365-b8bc-433f-80f7-65ff85deef54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.167948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035f365-b8bc-433f-80f7-65ff85deef54 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.168259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035f365-b8bc-433f-80f7-65ff85deef54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.168543 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035f365-b8bc-433f-80f7-65ff85deef54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.169643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035f365-b8bc-433f-80f7-65ff85deef54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.171389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035f365-b8bc-433f-80f7-65ff85deef54 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.183264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 786/1576] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:19.184285 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035f365-b8bc-433f-80f7-65ff85deef54 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.194809 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035f365-b8bc-433f-80f7-65ff85deef54 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.200813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0aed8-b540-486d-b7c9-b94d2c27471f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.201435 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0aed8-b540-486d-b7c9-b94d2c27471f admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.201714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0aed8-b540-486d-b7c9-b94d2c27471f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.201989 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0aed8-b540-486d-b7c9-b94d2c27471f admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.202641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0aed8-b540-486d-b7c9-b94d2c27471f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.204741 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0aed8-b540-486d-b7c9-b94d2c27471f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.207494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719439d-1389-4d01-a129-5da7d4deff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.208145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719439d-1389-4d01-a129-5da7d4deff06 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.208420 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719439d-1389-4d01-a129-5da7d4deff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.208701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719439d-1389-4d01-a129-5da7d4deff06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.209415 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719439d-1389-4d01-a129-5da7d4deff06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.211342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719439d-1389-4d01-a129-5da7d4deff06 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.250978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 787/1577] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:19 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:19.250978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0aed8-b540-486d-b7c9-b94d2c27471f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.254393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 791/1578] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:19 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:19.255715 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719439d-1389-4d01-a129-5da7d4deff06 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.325756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0aed8-b540-486d-b7c9-b94d2c27471f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.345692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719439d-1389-4d01-a129-5da7d4deff06 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.351596 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886fb4-2653-4766-b4cc-4361a7d41871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.352472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886fb4-2653-4766-b4cc-4361a7d41871 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.352749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886fb4-2653-4766-b4cc-4361a7d41871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.353022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886fb4-2653-4766-b4cc-4361a7d41871 admin admin] PATH_INFO: `/v3/projects/adf47975f465467498be24ebe2c81ad7/users/6d5a0cf37e3b4d3084a178f83642c144/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.372142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c886fb4-2653-4766-b4cc-4361a7d41871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf47975f465467498be24ebe2c81ad7, user_id=6d5a0cf37e3b4d3084a178f83642c144, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.373375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: WARNING py.warnings [None req-4c886fb4-2653-4766-b4cc-4361a7d41871 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:16:19.373375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: warnings.warn(msg) Aug 30 01:16:19.373375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: Aug 30 01:16:19.375378 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c886fb4-2653-4766-b4cc-4361a7d41871 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.393948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 792/1579] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:19 2022] PUT /identity/v3/projects/adf47975f465467498be24ebe2c81ad7/users/6d5a0cf37e3b4d3084a178f83642c144/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:19.395536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886fb4-2653-4766-b4cc-4361a7d41871 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.399301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1ffbf-bcd8-4e5a-8459-9721a27b9730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.400111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1ffbf-bcd8-4e5a-8459-9721a27b9730 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.400480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1ffbf-bcd8-4e5a-8459-9721a27b9730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.400795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1ffbf-bcd8-4e5a-8459-9721a27b9730 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.402766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1ffbf-bcd8-4e5a-8459-9721a27b9730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.404517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1ffbf-bcd8-4e5a-8459-9721a27b9730 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.431085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 788/1580] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:19 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:19.431085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1ffbf-bcd8-4e5a-8459-9721a27b9730 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.436901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:19.440474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1ffbf-bcd8-4e5a-8459-9721a27b9730 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.450171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:19.469331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886fb4-2653-4766-b4cc-4361a7d41871 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.474207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af453de3-597e-454f-adfd-8c6833197362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.474877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af453de3-597e-454f-adfd-8c6833197362 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.475143 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af453de3-597e-454f-adfd-8c6833197362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.475742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af453de3-597e-454f-adfd-8c6833197362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.476196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af453de3-597e-454f-adfd-8c6833197362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.477776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af453de3-597e-454f-adfd-8c6833197362 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.492871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 793/1581] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:19.492871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af453de3-597e-454f-adfd-8c6833197362 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.503132 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af453de3-597e-454f-adfd-8c6833197362 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.503696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8a61e-2040-4b79-be9b-a3fba0a6dd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.504226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8a61e-2040-4b79-be9b-a3fba0a6dd91 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.504529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8a61e-2040-4b79-be9b-a3fba0a6dd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.504803 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8a61e-2040-4b79-be9b-a3fba0a6dd91 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.505435 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8a61e-2040-4b79-be9b-a3fba0a6dd91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.507367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8a61e-2040-4b79-be9b-a3fba0a6dd91 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.507797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751b731-cbd0-4e56-b7cb-5c82ce34e520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.508419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751b731-cbd0-4e56-b7cb-5c82ce34e520 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.508724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751b731-cbd0-4e56-b7cb-5c82ce34e520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.508965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751b731-cbd0-4e56-b7cb-5c82ce34e520 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.509590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751b731-cbd0-4e56-b7cb-5c82ce34e520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.512984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751b731-cbd0-4e56-b7cb-5c82ce34e520 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.535162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 789/1582] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:19 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:19.536050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8a61e-2040-4b79-be9b-a3fba0a6dd91 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.543558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 794/1583] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:19 2022] POST /identity/v3/users => generated 303 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:19.544849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751b731-cbd0-4e56-b7cb-5c82ce34e520 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.550278 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8a61e-2040-4b79-be9b-a3fba0a6dd91 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.555297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8478692-e106-4c04-bd38-c6a41de22f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.555910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8478692-e106-4c04-bd38-c6a41de22f71 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.556212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8478692-e106-4c04-bd38-c6a41de22f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.556475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8478692-e106-4c04-bd38-c6a41de22f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.556711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751b731-cbd0-4e56-b7cb-5c82ce34e520 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.557132 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8478692-e106-4c04-bd38-c6a41de22f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.559493 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8478692-e106-4c04-bd38-c6a41de22f71 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.563949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4ce9e-d7b9-4e44-a99c-30ed859fd671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.564694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4ce9e-d7b9-4e44-a99c-30ed859fd671 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.565081 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4ce9e-d7b9-4e44-a99c-30ed859fd671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.565667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4ce9e-d7b9-4e44-a99c-30ed859fd671 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.565996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a4ce9e-d7b9-4e44-a99c-30ed859fd671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.567948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a4ce9e-d7b9-4e44-a99c-30ed859fd671 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.570850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 790/1584] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:19.571921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8478692-e106-4c04-bd38-c6a41de22f71 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.585812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8478692-e106-4c04-bd38-c6a41de22f71 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.592477 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7409ad2-5378-436d-b59c-f411244c901b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.593246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7409ad2-5378-436d-b59c-f411244c901b admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.593541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7409ad2-5378-436d-b59c-f411244c901b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.593806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7409ad2-5378-436d-b59c-f411244c901b admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.594618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 795/1585] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:19 2022] POST /identity/v3/users => generated 288 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:19.594818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7409ad2-5378-436d-b59c-f411244c901b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.595725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f587b82-ebec-4a91-a5b7-58e2ea4e4275 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.596034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f587b82-ebec-4a91-a5b7-58e2ea4e4275 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.596337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f587b82-ebec-4a91-a5b7-58e2ea4e4275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.597057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7409ad2-5378-436d-b59c-f411244c901b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.621105 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 791/1586] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:19 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:19.622663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7409ad2-5378-436d-b59c-f411244c901b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.623073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-4f587b82-ebec-4a91-a5b7-58e2ea4e4275 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:19.624662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4f587b82-ebec-4a91-a5b7-58e2ea4e4275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:19.634690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 796/1587] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:19.636888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f587b82-ebec-4a91-a5b7-58e2ea4e4275 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.637147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7409ad2-5378-436d-b59c-f411244c901b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.642151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e51cf-83d4-4365-abde-3017d563763e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.643140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e51cf-83d4-4365-abde-3017d563763e admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.643417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e51cf-83d4-4365-abde-3017d563763e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.643701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e51cf-83d4-4365-abde-3017d563763e admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.644386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e51cf-83d4-4365-abde-3017d563763e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.645977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e51cf-83d4-4365-abde-3017d563763e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.652564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f587b82-ebec-4a91-a5b7-58e2ea4e4275 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.657674 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0578e-3af6-4034-a51d-cbde878fde10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.658393 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 792/1588] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:19.658516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0578e-3af6-4034-a51d-cbde878fde10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.658613 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0578e-3af6-4034-a51d-cbde878fde10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.658866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0578e-3af6-4034-a51d-cbde878fde10 admin admin] PATH_INFO: `/v3/projects/705008e3e864488f897ebbf513d569c0/users/238d246c2cd64141a7be6fd56494d950/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.659833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e51cf-83d4-4365-abde-3017d563763e admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.673261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e51cf-83d4-4365-abde-3017d563763e admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.677382 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0578e-3af6-4034-a51d-cbde878fde10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=705008e3e864488f897ebbf513d569c0, user_id=238d246c2cd64141a7be6fd56494d950, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.679265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026aaef-af05-4a2a-8897-a5f2a4b931b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.679962 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026aaef-af05-4a2a-8897-a5f2a4b931b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.680132 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0578e-3af6-4034-a51d-cbde878fde10 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.680396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026aaef-af05-4a2a-8897-a5f2a4b931b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.680531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026aaef-af05-4a2a-8897-a5f2a4b931b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.681193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026aaef-af05-4a2a-8897-a5f2a4b931b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.686327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026aaef-af05-4a2a-8897-a5f2a4b931b0 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.697212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 797/1589] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:19 2022] PUT /identity/v3/projects/705008e3e864488f897ebbf513d569c0/users/238d246c2cd64141a7be6fd56494d950/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:19.698140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0578e-3af6-4034-a51d-cbde878fde10 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.708324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 793/1590] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:19.710076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026aaef-af05-4a2a-8897-a5f2a4b931b0 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.723704 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026aaef-af05-4a2a-8897-a5f2a4b931b0 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.772340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0578e-3af6-4034-a51d-cbde878fde10 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.777774 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86332c2-e8cc-4c36-ad6a-34646947fa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.778380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86332c2-e8cc-4c36-ad6a-34646947fa1c admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.778656 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86332c2-e8cc-4c36-ad6a-34646947fa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.778923 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86332c2-e8cc-4c36-ad6a-34646947fa1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.779560 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86332c2-e8cc-4c36-ad6a-34646947fa1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.781194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86332c2-e8cc-4c36-ad6a-34646947fa1c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.792743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 798/1591] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:19.793705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86332c2-e8cc-4c36-ad6a-34646947fa1c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.799117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c32c4-5c3b-4f7f-9088-c32269b8a682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.799791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c32c4-5c3b-4f7f-9088-c32269b8a682 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.800102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c32c4-5c3b-4f7f-9088-c32269b8a682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.800394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c32c4-5c3b-4f7f-9088-c32269b8a682 admin admin] PATH_INFO: `/v3/projects/3a9a8831e4f14b72b90d60854a9433ea/users/3a38f53091a04544a534184343bf0578/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.805462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86332c2-e8cc-4c36-ad6a-34646947fa1c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.810178 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef7863-04f4-4374-a803-4e3418e8b8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.810777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef7863-04f4-4374-a803-4e3418e8b8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.811045 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef7863-04f4-4374-a803-4e3418e8b8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.811308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef7863-04f4-4374-a803-4e3418e8b8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.811938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef7863-04f4-4374-a803-4e3418e8b8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.813556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef7863-04f4-4374-a803-4e3418e8b8d7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.816269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c32c4-5c3b-4f7f-9088-c32269b8a682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a9a8831e4f14b72b90d60854a9433ea, user_id=3a38f53091a04544a534184343bf0578, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.818943 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c32c4-5c3b-4f7f-9088-c32269b8a682 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.824712 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 799/1592] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:19.824712 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef7863-04f4-4374-a803-4e3418e8b8d7 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.832922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 794/1593] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:19 2022] PUT /identity/v3/projects/3a9a8831e4f14b72b90d60854a9433ea/users/3a38f53091a04544a534184343bf0578/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:19.844737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c32c4-5c3b-4f7f-9088-c32269b8a682 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.844858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef7863-04f4-4374-a803-4e3418e8b8d7 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.880661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:19.888347 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:19.901909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c32c4-5c3b-4f7f-9088-c32269b8a682 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.902552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a639e0-d4ae-458a-bf91-5206f1c4bebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.903169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a639e0-d4ae-458a-bf91-5206f1c4bebf admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.903450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a639e0-d4ae-458a-bf91-5206f1c4bebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.903717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a639e0-d4ae-458a-bf91-5206f1c4bebf admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.904408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a639e0-d4ae-458a-bf91-5206f1c4bebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.906015 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a639e0-d4ae-458a-bf91-5206f1c4bebf admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.906589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361baad-830f-4275-a7de-961c06ba90eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.907400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361baad-830f-4275-a7de-961c06ba90eb service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.907833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361baad-830f-4275-a7de-961c06ba90eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.908295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361baad-830f-4275-a7de-961c06ba90eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.916261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 800/1594] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:19.917905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a639e0-d4ae-458a-bf91-5206f1c4bebf admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.923191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361baad-830f-4275-a7de-961c06ba90eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.924755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361baad-830f-4275-a7de-961c06ba90eb service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.928137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a639e0-d4ae-458a-bf91-5206f1c4bebf admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.934499 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225119b-dd68-4c74-9b91-b6589569af91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.935109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225119b-dd68-4c74-9b91-b6589569af91 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.935274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 795/1595] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:19.936102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361baad-830f-4275-a7de-961c06ba90eb service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.936667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225119b-dd68-4c74-9b91-b6589569af91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.937038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225119b-dd68-4c74-9b91-b6589569af91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.937719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225119b-dd68-4c74-9b91-b6589569af91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.939457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225119b-dd68-4c74-9b91-b6589569af91 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.949280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361baad-830f-4275-a7de-961c06ba90eb service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.954835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 801/1596] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:19.954835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7f423-399c-49b7-ad5f-97696f556108 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.954835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7f423-399c-49b7-ad5f-97696f556108 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.954835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7f423-399c-49b7-ad5f-97696f556108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.958512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83dfe7-b5d8-4ac3-9d0e-8a3cce36a550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.959273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83dfe7-b5d8-4ac3-9d0e-8a3cce36a550 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.959610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83dfe7-b5d8-4ac3-9d0e-8a3cce36a550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.959916 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83dfe7-b5d8-4ac3-9d0e-8a3cce36a550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.960691 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83dfe7-b5d8-4ac3-9d0e-8a3cce36a550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:19.962631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83dfe7-b5d8-4ac3-9d0e-8a3cce36a550 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:19.978261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 796/1597] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:19.978261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83dfe7-b5d8-4ac3-9d0e-8a3cce36a550 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:19.989480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83dfe7-b5d8-4ac3-9d0e-8a3cce36a550 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:19.995352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805ca93-1368-4511-aa0c-27eb25b8c5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:19.996282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805ca93-1368-4511-aa0c-27eb25b8c5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:19.996554 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805ca93-1368-4511-aa0c-27eb25b8c5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:19.997102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805ca93-1368-4511-aa0c-27eb25b8c5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:19.997932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805ca93-1368-4511-aa0c-27eb25b8c5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:20.000027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805ca93-1368-4511-aa0c-27eb25b8c5b9 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:20.000784 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-2ed7f423-399c-49b7-ad5f-97696f556108 None None] MFA Rules not processed for user `238d246c2cd64141a7be6fd56494d950`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:20.001764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2ed7f423-399c-49b7-ad5f-97696f556108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:20.012930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 797/1598] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:20.014064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805ca93-1368-4511-aa0c-27eb25b8c5b9 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:20.025742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805ca93-1368-4511-aa0c-27eb25b8c5b9 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:20.032240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aff0f2-9dda-4809-a5d3-6299eff6bddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:20.033003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aff0f2-9dda-4809-a5d3-6299eff6bddb admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.033374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aff0f2-9dda-4809-a5d3-6299eff6bddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.033774 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aff0f2-9dda-4809-a5d3-6299eff6bddb admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.034987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aff0f2-9dda-4809-a5d3-6299eff6bddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:20.036977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aff0f2-9dda-4809-a5d3-6299eff6bddb admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:20.050662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 798/1599] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:20.050662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af5b3a-52d4-4c11-a90a-47314dd750db None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.050662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af5b3a-52d4-4c11-a90a-47314dd750db None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.050662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af5b3a-52d4-4c11-a90a-47314dd750db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.096935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-a4af5b3a-52d4-4c11-a90a-47314dd750db None None] MFA Rules not processed for user `3a38f53091a04544a534184343bf0578`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:20.097615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 802/1600] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:20.098248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a4af5b3a-52d4-4c11-a90a-47314dd750db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:20.098363 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7f423-399c-49b7-ad5f-97696f556108 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:20.115049 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7f423-399c-49b7-ad5f-97696f556108 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:20.120297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb797f0-dd49-4f5a-bff3-9038c5ecf4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:20.120942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb797f0-dd49-4f5a-bff3-9038c5ecf4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.121217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb797f0-dd49-4f5a-bff3-9038c5ecf4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.121483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb797f0-dd49-4f5a-bff3-9038c5ecf4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.122549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb797f0-dd49-4f5a-bff3-9038c5ecf4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:20.124519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb797f0-dd49-4f5a-bff3-9038c5ecf4e8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:20.138446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 803/1601] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:20.139307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb797f0-dd49-4f5a-bff3-9038c5ecf4e8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:20.152401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb797f0-dd49-4f5a-bff3-9038c5ecf4e8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:20.158848 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265c886-752a-4ae3-8cd2-b1c558632aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:20.159620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265c886-752a-4ae3-8cd2-b1c558632aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.159898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265c886-752a-4ae3-8cd2-b1c558632aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.160203 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265c886-752a-4ae3-8cd2-b1c558632aeb admin admin] PATH_INFO: `/v3/projects/c64e4eccf2c14dba96cbc1d8c4ab7831/users/e3352d04e04942aa87180eccc1597d52/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.178234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265c886-752a-4ae3-8cd2-b1c558632aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c64e4eccf2c14dba96cbc1d8c4ab7831, user_id=e3352d04e04942aa87180eccc1597d52, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:20.181087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265c886-752a-4ae3-8cd2-b1c558632aeb admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:20.198656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 799/1602] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:20.200254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af5b3a-52d4-4c11-a90a-47314dd750db None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:20.203934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 804/1603] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:20 2022] PUT /identity/v3/projects/c64e4eccf2c14dba96cbc1d8c4ab7831/users/e3352d04e04942aa87180eccc1597d52/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:20.204852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265c886-752a-4ae3-8cd2-b1c558632aeb admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:20.217305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af5b3a-52d4-4c11-a90a-47314dd750db None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:20.278431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265c886-752a-4ae3-8cd2-b1c558632aeb admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:20.283390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99881d-5ab4-43a2-860f-afded067d471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:20.284272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99881d-5ab4-43a2-860f-afded067d471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.284640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99881d-5ab4-43a2-860f-afded067d471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.285008 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99881d-5ab4-43a2-860f-afded067d471 admin admin] PATH_INFO: `/v3/projects/8526a71f7a514e7d9aa2635abdce933c/users/f02554c1bfbb446a85e351b8f87553a4/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.294516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71dcc0-c3b0-4d1f-9198-47513fc75ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:20.295640 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71dcc0-c3b0-4d1f-9198-47513fc75ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.295945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71dcc0-c3b0-4d1f-9198-47513fc75ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.296881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71dcc0-c3b0-4d1f-9198-47513fc75ff1 admin admin] PATH_INFO: `/v3/projects/a0afbc954ebc4555bfc59a1899d9f4aa/users/ae48c1e12a514e3c83b64204b7b4bc0d/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.314308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99881d-5ab4-43a2-860f-afded067d471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8526a71f7a514e7d9aa2635abdce933c, user_id=f02554c1bfbb446a85e351b8f87553a4, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:20.314633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71dcc0-c3b0-4d1f-9198-47513fc75ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0afbc954ebc4555bfc59a1899d9f4aa, user_id=ae48c1e12a514e3c83b64204b7b4bc0d, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:20.317770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71dcc0-c3b0-4d1f-9198-47513fc75ff1 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:20.318657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99881d-5ab4-43a2-860f-afded067d471 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:20.335162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 800/1604] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:20 2022] PUT /identity/v3/projects/a0afbc954ebc4555bfc59a1899d9f4aa/users/ae48c1e12a514e3c83b64204b7b4bc0d/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:20.336264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71dcc0-c3b0-4d1f-9198-47513fc75ff1 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:20.354861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 805/1605] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:20 2022] PUT /identity/v3/projects/8526a71f7a514e7d9aa2635abdce933c/users/f02554c1bfbb446a85e351b8f87553a4/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:20.356122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99881d-5ab4-43a2-860f-afded067d471 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:20.386676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:20.396459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:20.461307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71dcc0-c3b0-4d1f-9198-47513fc75ff1 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:20.468400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534be5e4-23af-4673-bb73-8c4c6ed9925f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:20.469869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534be5e4-23af-4673-bb73-8c4c6ed9925f admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.470196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99881d-5ab4-43a2-860f-afded067d471 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:20.470340 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534be5e4-23af-4673-bb73-8c4c6ed9925f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.470614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534be5e4-23af-4673-bb73-8c4c6ed9925f admin admin] PATH_INFO: `/v3/projects/a69f2b2876ed47459a8b960ff679c724/users/d5bd0cb311db42d7882a4cf90b0d85e8/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.478570 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0932aed-246e-46aa-8a9d-119f01554ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:20.479964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0932aed-246e-46aa-8a9d-119f01554ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.480500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0932aed-246e-46aa-8a9d-119f01554ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.480923 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0932aed-246e-46aa-8a9d-119f01554ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.481804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0932aed-246e-46aa-8a9d-119f01554ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:20.484378 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0932aed-246e-46aa-8a9d-119f01554ba3 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:20.492042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534be5e4-23af-4673-bb73-8c4c6ed9925f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a69f2b2876ed47459a8b960ff679c724, user_id=d5bd0cb311db42d7882a4cf90b0d85e8, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:20.496400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534be5e4-23af-4673-bb73-8c4c6ed9925f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:20.518428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 801/1606] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:20 2022] PUT /identity/v3/projects/a69f2b2876ed47459a8b960ff679c724/users/d5bd0cb311db42d7882a4cf90b0d85e8/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:20.519294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534be5e4-23af-4673-bb73-8c4c6ed9925f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:20.524624 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 806/1607] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:20 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:20.525466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0932aed-246e-46aa-8a9d-119f01554ba3 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:20.606847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534be5e4-23af-4673-bb73-8c4c6ed9925f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:20.627062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0932aed-246e-46aa-8a9d-119f01554ba3 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:20.633121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748141f-a95c-4f24-a262-02d9b471a775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:20.633734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748141f-a95c-4f24-a262-02d9b471a775 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.634013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748141f-a95c-4f24-a262-02d9b471a775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.634292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748141f-a95c-4f24-a262-02d9b471a775 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.634948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748141f-a95c-4f24-a262-02d9b471a775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:20.638243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748141f-a95c-4f24-a262-02d9b471a775 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:20.688596 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 807/1608] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:20 2022] POST /identity/v3/projects => generated 353 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:20.691787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee60e4-7568-466a-9ecb-19cb6f692861 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.692501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee60e4-7568-466a-9ecb-19cb6f692861 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.693487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee60e4-7568-466a-9ecb-19cb6f692861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.701625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3f79a-f265-44b3-96a8-373d6f7f31de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:20.702638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3f79a-f265-44b3-96a8-373d6f7f31de admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.703060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3f79a-f265-44b3-96a8-373d6f7f31de admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.704382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3f79a-f265-44b3-96a8-373d6f7f31de admin admin] PATH_INFO: `/v3/projects/9bf90b4e09e94cc984fe36b03b08a0b6/users/5b2e7c46b6aa4800834115196866b244/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.724523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3f79a-f265-44b3-96a8-373d6f7f31de admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf90b4e09e94cc984fe36b03b08a0b6, user_id=5b2e7c46b6aa4800834115196866b244, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:20.728243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3f79a-f265-44b3-96a8-373d6f7f31de admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:20.730613 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ddee60e4-7568-466a-9ecb-19cb6f692861 None None] MFA Rules not processed for user `e3352d04e04942aa87180eccc1597d52`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:20.731970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ddee60e4-7568-466a-9ecb-19cb6f692861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:20.749321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 802/1609] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:20 2022] PUT /identity/v3/projects/9bf90b4e09e94cc984fe36b03b08a0b6/users/5b2e7c46b6aa4800834115196866b244/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:20.750775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbf781-715f-4b2e-90e3-8f6adc1a9272 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.751061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbf781-715f-4b2e-90e3-8f6adc1a9272 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.751336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbf781-715f-4b2e-90e3-8f6adc1a9272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.795569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-77cbf781-715f-4b2e-90e3-8f6adc1a9272 None None] MFA Rules not processed for user `ae48c1e12a514e3c83b64204b7b4bc0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:20.797025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-77cbf781-715f-4b2e-90e3-8f6adc1a9272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:20.826219 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:20.831334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 808/1610] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:20.833573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8acce-2243-4482-8b34-716bb36374d7 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.833809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8acce-2243-4482-8b34-716bb36374d7 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.834188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8acce-2243-4482-8b34-716bb36374d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.898417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:20.906881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 803/1611] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:20.908310 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c241fb-d46b-4dd8-8724-d9dcfc377559 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:20.908593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c241fb-d46b-4dd8-8724-d9dcfc377559 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:20.908866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c241fb-d46b-4dd8-8724-d9dcfc377559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:20.921196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6bc8acce-2243-4482-8b34-716bb36374d7 None None] MFA Rules not processed for user `f02554c1bfbb446a85e351b8f87553a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:20.923915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6bc8acce-2243-4482-8b34-716bb36374d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:20.935265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-72c241fb-d46b-4dd8-8724-d9dcfc377559 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:20.936517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-72c241fb-d46b-4dd8-8724-d9dcfc377559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:21.010042 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 809/1612] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:20 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:21.011341 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 804/1613] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:21.012179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c241fb-d46b-4dd8-8724-d9dcfc377559 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.013557 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8acce-2243-4482-8b34-716bb36374d7 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.033695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8acce-2243-4482-8b34-716bb36374d7 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.036294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c241fb-d46b-4dd8-8724-d9dcfc377559 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.041016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca518cd9-139e-418d-aad4-96d402831ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.042318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca518cd9-139e-418d-aad4-96d402831ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.042723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca518cd9-139e-418d-aad4-96d402831ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.044544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca518cd9-139e-418d-aad4-96d402831ba0 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.046875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82743892-ee57-4109-a4e4-ad2fe7a7cc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.048391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82743892-ee57-4109-a4e4-ad2fe7a7cc8a admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.048714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82743892-ee57-4109-a4e4-ad2fe7a7cc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.049091 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82743892-ee57-4109-a4e4-ad2fe7a7cc8a admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.049740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82743892-ee57-4109-a4e4-ad2fe7a7cc8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.049966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca518cd9-139e-418d-aad4-96d402831ba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.051784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82743892-ee57-4109-a4e4-ad2fe7a7cc8a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.052528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca518cd9-139e-418d-aad4-96d402831ba0 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.084983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 810/1614] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:21 2022] POST /identity/v3/users => generated 294 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:21.085989 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 805/1615] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:21 2022] POST /identity/v3/users => generated 283 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:21.087408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931278de-fde7-45c9-b175-0a2de64e761b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.087667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30263d-297d-4842-af98-cd6510adb840 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.088011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30263d-297d-4842-af98-cd6510adb840 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.089035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30263d-297d-4842-af98-cd6510adb840 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.089257 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931278de-fde7-45c9-b175-0a2de64e761b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.089694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931278de-fde7-45c9-b175-0a2de64e761b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.106519 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 806/1616] 10.222.0.44 () {54 vars in 962 bytes} [Tue Aug 30 01:16:21 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:16:21.107488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30263d-297d-4842-af98-cd6510adb840 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.126105 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-931278de-fde7-45c9-b175-0a2de64e761b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:21.127274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-931278de-fde7-45c9-b175-0a2de64e761b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:21.136428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 811/1617] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:21.137492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931278de-fde7-45c9-b175-0a2de64e761b None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.149040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931278de-fde7-45c9-b175-0a2de64e761b None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.154580 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92c03f-0d0a-4d55-a149-25a753bbc554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.155578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92c03f-0d0a-4d55-a149-25a753bbc554 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.156148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92c03f-0d0a-4d55-a149-25a753bbc554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.156639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92c03f-0d0a-4d55-a149-25a753bbc554 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.157744 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92c03f-0d0a-4d55-a149-25a753bbc554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.159851 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92c03f-0d0a-4d55-a149-25a753bbc554 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.199451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 812/1618] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:21 2022] POST /identity/v3/projects => generated 403 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:21.200720 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92c03f-0d0a-4d55-a149-25a753bbc554 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.203434 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ea92c03f-0d0a-4d55-a149-25a753bbc554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:21.212113 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30263d-297d-4842-af98-cd6510adb840 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.290650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92c03f-0d0a-4d55-a149-25a753bbc554 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.296170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33965c00-6f13-44b2-b319-a7b06f406190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.296808 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33965c00-6f13-44b2-b319-a7b06f406190 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.297762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33965c00-6f13-44b2-b319-a7b06f406190 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.298044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33965c00-6f13-44b2-b319-a7b06f406190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.298203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a785ee-380b-43db-a6eb-7d6ca52f35c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.298731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a785ee-380b-43db-a6eb-7d6ca52f35c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.299007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a785ee-380b-43db-a6eb-7d6ca52f35c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.299300 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a785ee-380b-43db-a6eb-7d6ca52f35c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.320896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:21.411308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33965c00-6f13-44b2-b319-a7b06f406190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.412812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33965c00-6f13-44b2-b319-a7b06f406190 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.441928 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 813/1619] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:21.445021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33965c00-6f13-44b2-b319-a7b06f406190 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.453574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a785ee-380b-43db-a6eb-7d6ca52f35c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.455140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a785ee-380b-43db-a6eb-7d6ca52f35c0 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.462981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:21.472212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 807/1620] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:21.472840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33965c00-6f13-44b2-b319-a7b06f406190 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.473555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a785ee-380b-43db-a6eb-7d6ca52f35c0 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.489870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15679698-766f-4a40-ada1-0712e2272558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.490611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15679698-766f-4a40-ada1-0712e2272558 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.490735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a785ee-380b-43db-a6eb-7d6ca52f35c0 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.491146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15679698-766f-4a40-ada1-0712e2272558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.491493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15679698-766f-4a40-ada1-0712e2272558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.492444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15679698-766f-4a40-ada1-0712e2272558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.495496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15679698-766f-4a40-ada1-0712e2272558 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.503765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba41dce-a681-4937-ac57-d2b377302a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.504798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba41dce-a681-4937-ac57-d2b377302a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.505036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba41dce-a681-4937-ac57-d2b377302a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.505305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba41dce-a681-4937-ac57-d2b377302a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.505982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba41dce-a681-4937-ac57-d2b377302a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.513478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 814/1621] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:21.513812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba41dce-a681-4937-ac57-d2b377302a22 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.514011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608f14-1ae8-4bc9-9d52-565e2e8bc0e2 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.514011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608f14-1ae8-4bc9-9d52-565e2e8bc0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.514011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608f14-1ae8-4bc9-9d52-565e2e8bc0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.548969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 808/1622] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:21.550201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba41dce-a681-4937-ac57-d2b377302a22 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.566278 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba41dce-a681-4937-ac57-d2b377302a22 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.570966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877405a-1997-4bc2-ad5c-f3859f84b357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.571729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877405a-1997-4bc2-ad5c-f3859f84b357 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.571985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877405a-1997-4bc2-ad5c-f3859f84b357 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.572444 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877405a-1997-4bc2-ad5c-f3859f84b357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.586428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877405a-1997-4bc2-ad5c-f3859f84b357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.587873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877405a-1997-4bc2-ad5c-f3859f84b357 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.599152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-fb608f14-1ae8-4bc9-9d52-565e2e8bc0e2 None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:21.600488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 809/1623] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:21.601358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877405a-1997-4bc2-ad5c-f3859f84b357 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.602030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-fb608f14-1ae8-4bc9-9d52-565e2e8bc0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:21.613632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877405a-1997-4bc2-ad5c-f3859f84b357 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.624129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f824f8a-8fa1-4988-9aad-e4872f78bb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.624389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f824f8a-8fa1-4988-9aad-e4872f78bb1f admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.624792 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f824f8a-8fa1-4988-9aad-e4872f78bb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.625274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f824f8a-8fa1-4988-9aad-e4872f78bb1f admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.626417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f824f8a-8fa1-4988-9aad-e4872f78bb1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.630212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f824f8a-8fa1-4988-9aad-e4872f78bb1f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.656921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 810/1624] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:21 2022] POST /identity/v3/users => generated 308 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:21.658196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f824f8a-8fa1-4988-9aad-e4872f78bb1f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.671930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f824f8a-8fa1-4988-9aad-e4872f78bb1f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.677866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510cdcf0-1d36-4908-80c4-9d785f86ebea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.678797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cdcf0-1d36-4908-80c4-9d785f86ebea admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.679258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cdcf0-1d36-4908-80c4-9d785f86ebea admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.679577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cdcf0-1d36-4908-80c4-9d785f86ebea admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.680824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510cdcf0-1d36-4908-80c4-9d785f86ebea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.683526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510cdcf0-1d36-4908-80c4-9d785f86ebea admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.716923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 811/1625] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:21.717892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510cdcf0-1d36-4908-80c4-9d785f86ebea admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.747821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510cdcf0-1d36-4908-80c4-9d785f86ebea admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.766647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b14e7-b21f-48cd-a81f-df6e0dc73659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.767345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b14e7-b21f-48cd-a81f-df6e0dc73659 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.767887 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b14e7-b21f-48cd-a81f-df6e0dc73659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.771027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b14e7-b21f-48cd-a81f-df6e0dc73659 admin admin] PATH_INFO: `/v3/projects/ee48e31c633544009aeb6124fbb0bd4a/users/bb532d212e11401098c306e088719a52/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.806575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 815/1626] 10.222.0.44 () {58 vars in 1059 bytes} [Tue Aug 30 01:16:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:21.809692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb608f14-1ae8-4bc9-9d52-565e2e8bc0e2 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.842071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb608f14-1ae8-4bc9-9d52-565e2e8bc0e2 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.858276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cf966-dadc-42db-9965-47c27bb61a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.860511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cf966-dadc-42db-9965-47c27bb61a88 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.860778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cf966-dadc-42db-9965-47c27bb61a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.862136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cf966-dadc-42db-9965-47c27bb61a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.865746 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cf966-dadc-42db-9965-47c27bb61a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.865991 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b14e7-b21f-48cd-a81f-df6e0dc73659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee48e31c633544009aeb6124fbb0bd4a, user_id=bb532d212e11401098c306e088719a52, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.870850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cf966-dadc-42db-9965-47c27bb61a88 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.881488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b14e7-b21f-48cd-a81f-df6e0dc73659 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.914523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 816/1627] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:21.917544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 812/1628] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:21 2022] PUT /identity/v3/projects/ee48e31c633544009aeb6124fbb0bd4a/users/bb532d212e11401098c306e088719a52/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:21.918001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b14e7-b21f-48cd-a81f-df6e0dc73659 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.933730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cf966-dadc-42db-9965-47c27bb61a88 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.980634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cf966-dadc-42db-9965-47c27bb61a88 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:22.082351 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b14e7-b21f-48cd-a81f-df6e0dc73659 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:22.091783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf236e-955f-4669-a13a-9aeb5dbad400 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:22.092593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf236e-955f-4669-a13a-9aeb5dbad400 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:22.092879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf236e-955f-4669-a13a-9aeb5dbad400 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:22.093155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf236e-955f-4669-a13a-9aeb5dbad400 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:22.152237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:22.155155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef6427-3a62-4693-bbd9-79aaf20f2aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:22.155897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef6427-3a62-4693-bbd9-79aaf20f2aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:22.156320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef6427-3a62-4693-bbd9-79aaf20f2aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:22.156706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef6427-3a62-4693-bbd9-79aaf20f2aeb admin admin] PATH_INFO: `/v3/projects/75ad412645e24b9dbf7f38c6571108fd/users/b59f3575f9d7401b9f91f0d09d4a13be/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:22.190970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef6427-3a62-4693-bbd9-79aaf20f2aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ad412645e24b9dbf7f38c6571108fd, user_id=b59f3575f9d7401b9f91f0d09d4a13be, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:22.194011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef6427-3a62-4693-bbd9-79aaf20f2aeb admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:22.202608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf236e-955f-4669-a13a-9aeb5dbad400 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:22.206649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf236e-955f-4669-a13a-9aeb5dbad400 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:22.224652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 813/1629] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:22.225864 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf236e-955f-4669-a13a-9aeb5dbad400 service nova] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:22.227792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 817/1630] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:21 2022] PUT /identity/v3/projects/75ad412645e24b9dbf7f38c6571108fd/users/b59f3575f9d7401b9f91f0d09d4a13be/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:22.231511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef6427-3a62-4693-bbd9-79aaf20f2aeb admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:22.252575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf236e-955f-4669-a13a-9aeb5dbad400 service nova] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:22.425408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:22.451910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fcd82-dc6f-4777-b640-477a2ba8dc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:22.453587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fcd82-dc6f-4777-b640-477a2ba8dc74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:22.454026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fcd82-dc6f-4777-b640-477a2ba8dc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:22.454444 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fcd82-dc6f-4777-b640-477a2ba8dc74 admin admin] PATH_INFO: `/v3/projects/3fc9332258ce4a4c8ac41ae36a13011f/users/59bf8367e8ba47409e4463b0f83f10f7/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:22.481324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef6427-3a62-4693-bbd9-79aaf20f2aeb admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:22.484276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fcd82-dc6f-4777-b640-477a2ba8dc74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fc9332258ce4a4c8ac41ae36a13011f, user_id=59bf8367e8ba47409e4463b0f83f10f7, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:22.489419 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fcd82-dc6f-4777-b640-477a2ba8dc74 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:22.491686 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2c29b-0222-4e5e-a88f-2f326d52ea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:22.500064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c29b-0222-4e5e-a88f-2f326d52ea39 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:22.500987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c29b-0222-4e5e-a88f-2f326d52ea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:22.502606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c29b-0222-4e5e-a88f-2f326d52ea39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:22.503572 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2c29b-0222-4e5e-a88f-2f326d52ea39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:22.506298 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2c29b-0222-4e5e-a88f-2f326d52ea39 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:22.546695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 818/1631] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:22.548371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 814/1632] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:22 2022] PUT /identity/v3/projects/3fc9332258ce4a4c8ac41ae36a13011f/users/59bf8367e8ba47409e4463b0f83f10f7/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:22.548647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd657ef-73b5-4886-935d-c0e88c492ee0 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:22.548798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd657ef-73b5-4886-935d-c0e88c492ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:22.549170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd657ef-73b5-4886-935d-c0e88c492ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:22.553373 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fcd82-dc6f-4777-b640-477a2ba8dc74 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:22.629368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-2cd657ef-73b5-4886-935d-c0e88c492ee0 None None] MFA Rules not processed for user `b59f3575f9d7401b9f91f0d09d4a13be`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:22.632107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2cd657ef-73b5-4886-935d-c0e88c492ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:22.767698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fcd82-dc6f-4777-b640-477a2ba8dc74 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:22.786321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf9c6c-25f8-4584-83e9-92a53505bc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:22.787028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf9c6c-25f8-4584-83e9-92a53505bc2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:22.787307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf9c6c-25f8-4584-83e9-92a53505bc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:22.787579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf9c6c-25f8-4584-83e9-92a53505bc2d admin admin] PATH_INFO: `/v3/projects/ee48e31c633544009aeb6124fbb0bd4a/users/bb532d212e11401098c306e088719a52/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:22.805375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf9c6c-25f8-4584-83e9-92a53505bc2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee48e31c633544009aeb6124fbb0bd4a, user_id=bb532d212e11401098c306e088719a52, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:22.809524 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf9c6c-25f8-4584-83e9-92a53505bc2d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:22.833815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 819/1633] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:22.836673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd657ef-73b5-4886-935d-c0e88c492ee0 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:22.860998 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 815/1634] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:22 2022] PUT /identity/v3/projects/ee48e31c633544009aeb6124fbb0bd4a/users/bb532d212e11401098c306e088719a52/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:16:22.860998 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5b919-d714-49cb-8dd0-f7e070fcd5b4 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:22.860998 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5b919-d714-49cb-8dd0-f7e070fcd5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:22.860998 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5b919-d714-49cb-8dd0-f7e070fcd5b4 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:22.867028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd657ef-73b5-4886-935d-c0e88c492ee0 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:22.872096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 816/1635] 10.222.0.44 () {54 vars in 912 bytes} [Tue Aug 30 01:16:22 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:16:22.873053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5b919-d714-49cb-8dd0-f7e070fcd5b4 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:23.024068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbe9c6-409c-4328-ae5d-cddafaecfb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:23.025929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbe9c6-409c-4328-ae5d-cddafaecfb3a admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:23.026763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbe9c6-409c-4328-ae5d-cddafaecfb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:23.027423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbe9c6-409c-4328-ae5d-cddafaecfb3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:23.029224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbe9c6-409c-4328-ae5d-cddafaecfb3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:23.038018 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbe9c6-409c-4328-ae5d-cddafaecfb3a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:23.039635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:23.057654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 820/1636] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:23.060542 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1668f50-5192-432d-8e00-e2ec9d36ae56 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:23.060827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1668f50-5192-432d-8e00-e2ec9d36ae56 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:23.061094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1668f50-5192-432d-8e00-e2ec9d36ae56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:23.083722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5b919-d714-49cb-8dd0-f7e070fcd5b4 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:23.105510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5d11c-9b4f-4083-8ed5-77695ca99990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:23.106310 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5d11c-9b4f-4083-8ed5-77695ca99990 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:23.106606 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5d11c-9b4f-4083-8ed5-77695ca99990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:23.106870 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5d11c-9b4f-4083-8ed5-77695ca99990 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:23.107552 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5d11c-9b4f-4083-8ed5-77695ca99990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:23.113499 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5d11c-9b4f-4083-8ed5-77695ca99990 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:23.133358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d1668f50-5192-432d-8e00-e2ec9d36ae56 None None] MFA Rules not processed for user `bb532d212e11401098c306e088719a52`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:23.134665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d1668f50-5192-432d-8e00-e2ec9d36ae56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:23.185316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 817/1637] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:22 2022] POST /identity/v3/projects => generated 385 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:23.187294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcdfdc-6236-4a65-80d8-047812bd1ef3 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:23.187538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcdfdc-6236-4a65-80d8-047812bd1ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:23.188245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcdfdc-6236-4a65-80d8-047812bd1ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:23.243885 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-f8bcdfdc-6236-4a65-80d8-047812bd1ef3 None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:23.248107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f8bcdfdc-6236-4a65-80d8-047812bd1ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:23.253216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:23.293927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 821/1638] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:23 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:23.294895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1668f50-5192-432d-8e00-e2ec9d36ae56 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:23.313567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1668f50-5192-432d-8e00-e2ec9d36ae56 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:23.426177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 818/1639] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:16:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:23.427472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcdfdc-6236-4a65-80d8-047812bd1ef3 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:23.442902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02a0df-bb36-4acf-91bc-6c65c5424354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:23.444010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02a0df-bb36-4acf-91bc-6c65c5424354 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:23.444010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02a0df-bb36-4acf-91bc-6c65c5424354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:23.446794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02a0df-bb36-4acf-91bc-6c65c5424354 admin admin] PATH_INFO: `/v3/projects/3fc9332258ce4a4c8ac41ae36a13011f/users/59bf8367e8ba47409e4463b0f83f10f7/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:23.449977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02a0df-bb36-4acf-91bc-6c65c5424354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fc9332258ce4a4c8ac41ae36a13011f, user_id=59bf8367e8ba47409e4463b0f83f10f7, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:23.454159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02a0df-bb36-4acf-91bc-6c65c5424354 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:23.476150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 822/1640] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:23 2022] PUT /identity/v3/projects/3fc9332258ce4a4c8ac41ae36a13011f/users/59bf8367e8ba47409e4463b0f83f10f7/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:23.476150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02a0df-bb36-4acf-91bc-6c65c5424354 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:23.559857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcdfdc-6236-4a65-80d8-047812bd1ef3 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:23.656281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02a0df-bb36-4acf-91bc-6c65c5424354 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:23.670150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c58883-1713-4b4d-9e13-05fbf98630bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:23.670902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c58883-1713-4b4d-9e13-05fbf98630bb admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:23.671230 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c58883-1713-4b4d-9e13-05fbf98630bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:23.671574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c58883-1713-4b4d-9e13-05fbf98630bb admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:23.672270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c58883-1713-4b4d-9e13-05fbf98630bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:23.674034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1adff3-7bb9-4e19-95aa-4b9acda760da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:23.675524 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c58883-1713-4b4d-9e13-05fbf98630bb admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:23.676026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1adff3-7bb9-4e19-95aa-4b9acda760da service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:23.676549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1adff3-7bb9-4e19-95aa-4b9acda760da service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:23.677032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1adff3-7bb9-4e19-95aa-4b9acda760da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:23.723368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 819/1641] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:23 2022] POST /identity/v3/users => generated 299 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:23.723488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c58883-1713-4b4d-9e13-05fbf98630bb admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:23.740702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:23.749594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c58883-1713-4b4d-9e13-05fbf98630bb admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:23.757608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ac070-6fdb-4906-ae38-54c7cbae04dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:23.758251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ac070-6fdb-4906-ae38-54c7cbae04dc service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:23.758531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ac070-6fdb-4906-ae38-54c7cbae04dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:23.758801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ac070-6fdb-4906-ae38-54c7cbae04dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:23.827310 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1adff3-7bb9-4e19-95aa-4b9acda760da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:23.829653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1adff3-7bb9-4e19-95aa-4b9acda760da service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:23.858918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 823/1642] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:23 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:23.861625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f71f6-d7b8-46b9-a10f-a24ff98f6386 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:23.861907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f71f6-d7b8-46b9-a10f-a24ff98f6386 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:23.862175 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f71f6-d7b8-46b9-a10f-a24ff98f6386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:23.970693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-2f1f71f6-d7b8-46b9-a10f-a24ff98f6386 None None] MFA Rules not processed for user `59bf8367e8ba47409e4463b0f83f10f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:23.973598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2f1f71f6-d7b8-46b9-a10f-a24ff98f6386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:24.002322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ac070-6fdb-4906-ae38-54c7cbae04dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:24.012152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ac070-6fdb-4906-ae38-54c7cbae04dc service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:24.021976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:24.065943 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 820/1643] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:24.068992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ac070-6fdb-4906-ae38-54c7cbae04dc service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:24.090032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 824/1644] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:23 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:24.124711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ac070-6fdb-4906-ae38-54c7cbae04dc service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:24.148711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc607d7-1981-4527-bf3d-12d61339d78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:24.148711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc607d7-1981-4527-bf3d-12d61339d78a admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:24.148711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc607d7-1981-4527-bf3d-12d61339d78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:24.148711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc607d7-1981-4527-bf3d-12d61339d78a admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:24.149248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc607d7-1981-4527-bf3d-12d61339d78a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:24.153021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc607d7-1981-4527-bf3d-12d61339d78a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:24.181687 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 821/1645] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:24.183591 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f71f6-d7b8-46b9-a10f-a24ff98f6386 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:24.204111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f71f6-d7b8-46b9-a10f-a24ff98f6386 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:24.215412 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16907e15-8330-4785-8b18-f1c89cabe9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:24.217284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16907e15-8330-4785-8b18-f1c89cabe9cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:24.217590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16907e15-8330-4785-8b18-f1c89cabe9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:24.219131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16907e15-8330-4785-8b18-f1c89cabe9cc admin admin] PATH_INFO: `/v3/projects/6f5127a35f8c472fbdac4f5c0d784029/users/77eafa25e3f542aebe2df9eed440888b/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:24.253879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16907e15-8330-4785-8b18-f1c89cabe9cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5127a35f8c472fbdac4f5c0d784029, user_id=77eafa25e3f542aebe2df9eed440888b, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:24.256603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16907e15-8330-4785-8b18-f1c89cabe9cc admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:24.289453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc607d7-1981-4527-bf3d-12d61339d78a admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:24.295261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 825/1646] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:24 2022] PUT /identity/v3/projects/6f5127a35f8c472fbdac4f5c0d784029/users/77eafa25e3f542aebe2df9eed440888b/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:24.297741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16907e15-8330-4785-8b18-f1c89cabe9cc admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:24.357192 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc607d7-1981-4527-bf3d-12d61339d78a admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:24.498493 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d324e-9ba1-43b6-84c9-5b69969be5c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:24.498775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d324e-9ba1-43b6-84c9-5b69969be5c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:24.499012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d324e-9ba1-43b6-84c9-5b69969be5c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:24.499212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d324e-9ba1-43b6-84c9-5b69969be5c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:24.549714 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16907e15-8330-4785-8b18-f1c89cabe9cc admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:24.567888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f46ff9-02e9-4aa2-a589-11bf19ef265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:24.567888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f46ff9-02e9-4aa2-a589-11bf19ef265e admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:24.567888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f46ff9-02e9-4aa2-a589-11bf19ef265e admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:24.567888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f46ff9-02e9-4aa2-a589-11bf19ef265e admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:24.568701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f46ff9-02e9-4aa2-a589-11bf19ef265e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:24.570441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f46ff9-02e9-4aa2-a589-11bf19ef265e admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:24.618551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 826/1647] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:24.619619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f46ff9-02e9-4aa2-a589-11bf19ef265e admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:24.653973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f46ff9-02e9-4aa2-a589-11bf19ef265e admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:24.657314 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d324e-9ba1-43b6-84c9-5b69969be5c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:24.658795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d324e-9ba1-43b6-84c9-5b69969be5c6 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:24.680089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 822/1648] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:24 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:24.680827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a5812-11ce-4024-b7cc-bd47b8ca53dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:24.681526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a5812-11ce-4024-b7cc-bd47b8ca53dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:24.681810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a5812-11ce-4024-b7cc-bd47b8ca53dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:24.682160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a5812-11ce-4024-b7cc-bd47b8ca53dd admin admin] PATH_INFO: `/v3/projects/6f5127a35f8c472fbdac4f5c0d784029/users/77eafa25e3f542aebe2df9eed440888b/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:24.690073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a5812-11ce-4024-b7cc-bd47b8ca53dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5127a35f8c472fbdac4f5c0d784029, user_id=77eafa25e3f542aebe2df9eed440888b, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:24.693194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a5812-11ce-4024-b7cc-bd47b8ca53dd admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:24.724713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 827/1649] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:24 2022] PUT /identity/v3/projects/6f5127a35f8c472fbdac4f5c0d784029/users/77eafa25e3f542aebe2df9eed440888b/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:24.742043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34eba-1b4e-421a-9bf0-b644ef7caba7 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:24.743457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34eba-1b4e-421a-9bf0-b644ef7caba7 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:24.743768 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34eba-1b4e-421a-9bf0-b644ef7caba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:24.809843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a5812-11ce-4024-b7cc-bd47b8ca53dd admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:24.869323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-dbb34eba-1b4e-421a-9bf0-b644ef7caba7 None None] MFA Rules not processed for user `77eafa25e3f542aebe2df9eed440888b`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:24.872385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-dbb34eba-1b4e-421a-9bf0-b644ef7caba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:24.934828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:24.964761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a5812-11ce-4024-b7cc-bd47b8ca53dd admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:24.975763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfc53b-9909-4878-bc8a-6a5a99e87608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:24.975944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfc53b-9909-4878-bc8a-6a5a99e87608 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:24.976244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfc53b-9909-4878-bc8a-6a5a99e87608 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:24.976520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfc53b-9909-4878-bc8a-6a5a99e87608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:25.070221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:25.082034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 823/1650] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:25.149347 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2271b-5654-47e8-be5d-7116d077ae0c None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:25.150028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2271b-5654-47e8-be5d-7116d077ae0c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:25.150028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2271b-5654-47e8-be5d-7116d077ae0c None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:25.153153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 824/1651] 10.222.0.44 () {54 vars in 962 bytes} [Tue Aug 30 01:16:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:16:25.172996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfc53b-9909-4878-bc8a-6a5a99e87608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:25.173569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2e44d-9abd-428d-a540-40c7ff72886f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:25.173657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2e44d-9abd-428d-a540-40c7ff72886f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:25.173916 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2e44d-9abd-428d-a540-40c7ff72886f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:25.176597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfc53b-9909-4878-bc8a-6a5a99e87608 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:25.215549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 828/1652] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:25.232522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-f1b2e44d-9abd-428d-a540-40c7ff72886f None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:25.233683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f1b2e44d-9abd-428d-a540-40c7ff72886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:25.415296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 825/1653] 10.222.0.44 () {58 vars in 1059 bytes} [Tue Aug 30 01:16:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:25.421366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70643ef7-a421-4d9d-b2d3-917314363d35 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:25.421875 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70643ef7-a421-4d9d-b2d3-917314363d35 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:25.422893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70643ef7-a421-4d9d-b2d3-917314363d35 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:25.424623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 829/1654] 10.222.0.44 () {54 vars in 912 bytes} [Tue Aug 30 01:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:16:25.432123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae09691-212c-4bcd-adea-da70b551c799 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:25.432450 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae09691-212c-4bcd-adea-da70b551c799 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:25.432662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae09691-212c-4bcd-adea-da70b551c799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:25.470088 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-6ae09691-212c-4bcd-adea-da70b551c799 None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:25.473558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6ae09691-212c-4bcd-adea-da70b551c799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:25.497389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 826/1655] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:16:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:25.516448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70643ef7-a421-4d9d-b2d3-917314363d35 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:25.556745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70643ef7-a421-4d9d-b2d3-917314363d35 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:25.574510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0c9a1-3474-402e-a892-66726ec73074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:25.575155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0c9a1-3474-402e-a892-66726ec73074 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:25.575430 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0c9a1-3474-402e-a892-66726ec73074 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:25.576398 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0c9a1-3474-402e-a892-66726ec73074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:25.609007 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0c9a1-3474-402e-a892-66726ec73074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:25.610393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0c9a1-3474-402e-a892-66726ec73074 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:25.648929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 830/1656] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:25.720228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae09691-212c-4bcd-adea-da70b551c799 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:25.745930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae09691-212c-4bcd-adea-da70b551c799 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:25.755660 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d8888-10e5-49b5-b7e6-436402e1018a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:25.757211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d8888-10e5-49b5-b7e6-436402e1018a service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:25.757613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d8888-10e5-49b5-b7e6-436402e1018a service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:25.758006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d8888-10e5-49b5-b7e6-436402e1018a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:25.785717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d8888-10e5-49b5-b7e6-436402e1018a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:25.787229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d8888-10e5-49b5-b7e6-436402e1018a service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:25.840633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 827/1657] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:26.081489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0c9a1-3474-402e-a892-66726ec73074 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:26.323088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0c9a1-3474-402e-a892-66726ec73074 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:26.344735 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:26.345807 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:26.346384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:26.347557 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] PATH_INFO: `/v3/users/f02554c1bfbb446a85e351b8f87553a4` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:26.361427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f02554c1bfbb446a85e351b8f87553a4)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:26.363400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: WARNING py.warnings [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:16:26.363400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: warnings.warn(msg) Aug 30 01:16:26.363400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: Aug 30 01:16:26.365963 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:26.627558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f02554c1bfbb446a85e351b8f87553a4'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:26.722540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f02554c1bfbb446a85e351b8f87553a4'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:26.768382 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f02554c1bfbb446a85e351b8f87553a4'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:26.789405 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f02554c1bfbb446a85e351b8f87553a4'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:26.790691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f02554c1bfbb446a85e351b8f87553a4'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:26.815176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 831/1658] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:16:26 2022] DELETE /identity/v3/users/f02554c1bfbb446a85e351b8f87553a4 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:26.822903 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abd120-4f1a-4f30-a662-8bff42f2db07 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:26.823508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abd120-4f1a-4f30-a662-8bff42f2db07 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:26.824826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abd120-4f1a-4f30-a662-8bff42f2db07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:26.924122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c3abd120-4f1a-4f30-a662-8bff42f2db07 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:26.929144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c3abd120-4f1a-4f30-a662-8bff42f2db07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:27.243384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 828/1659] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:27.887678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:27.889374 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:28.047444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:28.109341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757818c-1ce5-4191-9aad-b78ee2151eb1 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:28.121607 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baf0a7-1795-49fa-86d1-3e8d58f692f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:28.123455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baf0a7-1795-49fa-86d1-3e8d58f692f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:28.124860 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baf0a7-1795-49fa-86d1-3e8d58f692f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:28.125596 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baf0a7-1795-49fa-86d1-3e8d58f692f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:28.144218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59baf0a7-1795-49fa-86d1-3e8d58f692f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:28.147395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59baf0a7-1795-49fa-86d1-3e8d58f692f1 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:28.191351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 832/1660] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:28.565442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abd120-4f1a-4f30-a662-8bff42f2db07 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:28.567965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c3abd120-4f1a-4f30-a662-8bff42f2db07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:28.654921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:28.783226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abd120-4f1a-4f30-a662-8bff42f2db07 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:28.791204 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a412d-b2eb-4216-a841-cf5295689832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:28.793702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a412d-b2eb-4216-a841-cf5295689832 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:28.794146 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a412d-b2eb-4216-a841-cf5295689832 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:28.794558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a412d-b2eb-4216-a841-cf5295689832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:28.799908 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b05a412d-b2eb-4216-a841-cf5295689832 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:29.007202 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a412d-b2eb-4216-a841-cf5295689832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:29.009056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a412d-b2eb-4216-a841-cf5295689832 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:29.037294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 829/1661] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:30.200015 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baf0a7-1795-49fa-86d1-3e8d58f692f1 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:30.203916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-59baf0a7-1795-49fa-86d1-3e8d58f692f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:30.251419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baf0a7-1795-49fa-86d1-3e8d58f692f1 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:30.276172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:30.277333 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:30.277824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:30.278304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] PATH_INFO: `/v3/projects/8526a71f7a514e7d9aa2635abdce933c` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:30.288956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8526a71f7a514e7d9aa2635abdce933c)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:30.293420 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:30.437509 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8526a71f7a514e7d9aa2635abdce933c', 'request_id': 'req-a5763a29-5469-4b34-9ca6-85025b5f328a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:30.470362 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8526a71f7a514e7d9aa2635abdce933c', 'request_id': 'req-a5763a29-5469-4b34-9ca6-85025b5f328a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:30.505834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8526a71f7a514e7d9aa2635abdce933c', 'request_id': 'req-a5763a29-5469-4b34-9ca6-85025b5f328a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:30.538850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] The token cache is being invalidate because project 8526a71f7a514e7d9aa2635abdce933c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:16:30.539164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:30.540356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:30.545556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 833/1662] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:16:30 2022] DELETE /identity/v3/projects/8526a71f7a514e7d9aa2635abdce933c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:30.551946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a412d-b2eb-4216-a841-cf5295689832 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:30.633558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a412d-b2eb-4216-a841-cf5295689832 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:30.836756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371daccb-e281-48c3-b907-b8b2bba703b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:30.841018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371daccb-e281-48c3-b907-b8b2bba703b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:30.841586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371daccb-e281-48c3-b907-b8b2bba703b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:30.842019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371daccb-e281-48c3-b907-b8b2bba703b6 admin admin] PATH_INFO: `/v3/users/59bf8367e8ba47409e4463b0f83f10f7` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:30.843602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371daccb-e281-48c3-b907-b8b2bba703b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59bf8367e8ba47409e4463b0f83f10f7)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:30.847621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371daccb-e281-48c3-b907-b8b2bba703b6 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:31.041273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-371daccb-e281-48c3-b907-b8b2bba703b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59bf8367e8ba47409e4463b0f83f10f7'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:31.074837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-371daccb-e281-48c3-b907-b8b2bba703b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59bf8367e8ba47409e4463b0f83f10f7'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:31.092624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-371daccb-e281-48c3-b907-b8b2bba703b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59bf8367e8ba47409e4463b0f83f10f7'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:31.096072 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-371daccb-e281-48c3-b907-b8b2bba703b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59bf8367e8ba47409e4463b0f83f10f7'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:31.125477 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-371daccb-e281-48c3-b907-b8b2bba703b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59bf8367e8ba47409e4463b0f83f10f7'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:31.144552 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:31.163925 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 830/1663] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:16:30 2022] DELETE /identity/v3/users/59bf8367e8ba47409e4463b0f83f10f7 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:32.748226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:32.749806 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:32.801991 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5763a29-5469-4b34-9ca6-85025b5f328a admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:32.971838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f1391-656d-4370-a212-078159d2c00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:32.984938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f1391-656d-4370-a212-078159d2c00f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:32.989317 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f1391-656d-4370-a212-078159d2c00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:32.989594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f1391-656d-4370-a212-078159d2c00f admin admin] PATH_INFO: `/v3/projects/3fc9332258ce4a4c8ac41ae36a13011f` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:32.992124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f1391-656d-4370-a212-078159d2c00f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fc9332258ce4a4c8ac41ae36a13011f)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:32.998952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f1391-656d-4370-a212-078159d2c00f admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:33.170165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-702f1391-656d-4370-a212-078159d2c00f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fc9332258ce4a4c8ac41ae36a13011f', 'request_id': 'req-702f1391-656d-4370-a212-078159d2c00f'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:33.196731 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-702f1391-656d-4370-a212-078159d2c00f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fc9332258ce4a4c8ac41ae36a13011f', 'request_id': 'req-702f1391-656d-4370-a212-078159d2c00f'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:33.208676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-702f1391-656d-4370-a212-078159d2c00f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fc9332258ce4a4c8ac41ae36a13011f', 'request_id': 'req-702f1391-656d-4370-a212-078159d2c00f'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:33.212843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:33.237415 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-702f1391-656d-4370-a212-078159d2c00f admin admin] The token cache is being invalidate because project 3fc9332258ce4a4c8ac41ae36a13011f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:16:33.237966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-702f1391-656d-4370-a212-078159d2c00f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:33.238830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-702f1391-656d-4370-a212-078159d2c00f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:33.241098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 834/1664] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:16:32 2022] DELETE /identity/v3/projects/3fc9332258ce4a4c8ac41ae36a13011f => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:33.247549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c9864-08f5-4a26-85d0-800f9a4e3201 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:33.248127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c9864-08f5-4a26-85d0-800f9a4e3201 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:33.248539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c9864-08f5-4a26-85d0-800f9a4e3201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:33.292847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-385c9864-08f5-4a26-85d0-800f9a4e3201 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:33.294116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-385c9864-08f5-4a26-85d0-800f9a4e3201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:33.393737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aaa2ef-d871-4bdf-8d9c-d74bffba9991 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:33.394019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aaa2ef-d871-4bdf-8d9c-d74bffba9991 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:33.394302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aaa2ef-d871-4bdf-8d9c-d74bffba9991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:33.490417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-38aaa2ef-d871-4bdf-8d9c-d74bffba9991 None None] MFA Rules not processed for user `d5bd0cb311db42d7882a4cf90b0d85e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:33.493176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-38aaa2ef-d871-4bdf-8d9c-d74bffba9991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:33.509510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 831/1665] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:33.516434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce815fc-b1fb-4aae-8c9c-d173876c7099 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:33.517226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce815fc-b1fb-4aae-8c9c-d173876c7099 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:33.525844 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce815fc-b1fb-4aae-8c9c-d173876c7099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:33.592128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-7ce815fc-b1fb-4aae-8c9c-d173876c7099 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:33.593308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7ce815fc-b1fb-4aae-8c9c-d173876c7099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:33.610418 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 832/1666] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:33.615891 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce815fc-b1fb-4aae-8c9c-d173876c7099 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:33.642205 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce815fc-b1fb-4aae-8c9c-d173876c7099 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:33.654025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76458ac7-87eb-4e0b-a88f-eb3830f01685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:33.654702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76458ac7-87eb-4e0b-a88f-eb3830f01685 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:33.655020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76458ac7-87eb-4e0b-a88f-eb3830f01685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:33.657172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76458ac7-87eb-4e0b-a88f-eb3830f01685 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:33.659190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76458ac7-87eb-4e0b-a88f-eb3830f01685 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:33.659878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76458ac7-87eb-4e0b-a88f-eb3830f01685 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:33.663561 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76458ac7-87eb-4e0b-a88f-eb3830f01685 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:33.701849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 833/1667] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:33.702818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76458ac7-87eb-4e0b-a88f-eb3830f01685 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:33.732370 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76458ac7-87eb-4e0b-a88f-eb3830f01685 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:33.744488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaea753-59c2-4eae-a2a4-d30b12ec8718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:33.745381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaea753-59c2-4eae-a2a4-d30b12ec8718 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:33.745662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaea753-59c2-4eae-a2a4-d30b12ec8718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:33.745927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaea753-59c2-4eae-a2a4-d30b12ec8718 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:33.746846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaea753-59c2-4eae-a2a4-d30b12ec8718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:33.751771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaea753-59c2-4eae-a2a4-d30b12ec8718 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:33.753650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 835/1668] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:33.759929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aaa2ef-d871-4bdf-8d9c-d74bffba9991 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:33.762872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-38aaa2ef-d871-4bdf-8d9c-d74bffba9991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:33.824757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 834/1669] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:33 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:33.829024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaea753-59c2-4eae-a2a4-d30b12ec8718 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:34.025884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aaa2ef-d871-4bdf-8d9c-d74bffba9991 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:34.060561 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaea753-59c2-4eae-a2a4-d30b12ec8718 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:34.071680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3df9c-9b3f-4f03-b16d-baf9b34aedfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:34.074486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3df9c-9b3f-4f03-b16d-baf9b34aedfd admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:34.077165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3df9c-9b3f-4f03-b16d-baf9b34aedfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:34.077493 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3df9c-9b3f-4f03-b16d-baf9b34aedfd admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:34.078781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3df9c-9b3f-4f03-b16d-baf9b34aedfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:34.084100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3df9c-9b3f-4f03-b16d-baf9b34aedfd admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:34.163296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 835/1670] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:33 2022] POST /identity/v3/users => generated 283 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:16:34.167820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66fe35-2547-449f-8ef8-ac5de8d3410a None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:34.168197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf82ef1-c422-41c2-b56b-de185550e878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:34.168354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66fe35-2547-449f-8ef8-ac5de8d3410a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:34.168354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66fe35-2547-449f-8ef8-ac5de8d3410a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:34.168702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf82ef1-c422-41c2-b56b-de185550e878 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:34.169024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf82ef1-c422-41c2-b56b-de185550e878 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:34.169252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf82ef1-c422-41c2-b56b-de185550e878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:34.208186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:34.244923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8b66fe35-2547-449f-8ef8-ac5de8d3410a None None] MFA Rules not processed for user `6d5a0cf37e3b4d3084a178f83642c144`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:34.247701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8b66fe35-2547-449f-8ef8-ac5de8d3410a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:34.303576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf82ef1-c422-41c2-b56b-de185550e878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:34.305234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf82ef1-c422-41c2-b56b-de185550e878 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:34.332145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 836/1671] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:34.333072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf82ef1-c422-41c2-b56b-de185550e878 service nova] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:34.355750 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf82ef1-c422-41c2-b56b-de185550e878 service nova] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:34.375303 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a4ca4-11e7-468e-ae9a-3dfbc9ffc902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:34.375943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a4ca4-11e7-468e-ae9a-3dfbc9ffc902 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:34.376272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a4ca4-11e7-468e-ae9a-3dfbc9ffc902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:34.376550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a4ca4-11e7-468e-ae9a-3dfbc9ffc902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:34.377214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a4ca4-11e7-468e-ae9a-3dfbc9ffc902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:34.378858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a4ca4-11e7-468e-ae9a-3dfbc9ffc902 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:34.404220 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:34.407184 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 837/1672] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:34.411535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a4ca4-11e7-468e-ae9a-3dfbc9ffc902 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:34.440646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a4ca4-11e7-468e-ae9a-3dfbc9ffc902 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:34.452314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d5ac0-8859-4b2a-9eb4-266202af6501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:34.452314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d5ac0-8859-4b2a-9eb4-266202af6501 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:34.457233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d5ac0-8859-4b2a-9eb4-266202af6501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:34.468349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d5ac0-8859-4b2a-9eb4-266202af6501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:34.469352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d5ac0-8859-4b2a-9eb4-266202af6501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:34.473726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d5ac0-8859-4b2a-9eb4-266202af6501 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:34.476266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 836/1673] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:34.487723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb473370-f6e8-4e97-9183-d0031495f80a None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:34.488025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb473370-f6e8-4e97-9183-d0031495f80a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:34.488350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb473370-f6e8-4e97-9183-d0031495f80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:34.491865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 838/1674] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:16:34.497505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d5ac0-8859-4b2a-9eb4-266202af6501 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:34.539392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-bb473370-f6e8-4e97-9183-d0031495f80a None None] MFA Rules not processed for user `6d5a0cf37e3b4d3084a178f83642c144`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:34.539926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-bb473370-f6e8-4e97-9183-d0031495f80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:34.544869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d5ac0-8859-4b2a-9eb4-266202af6501 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:34.558524 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6c430-ac2b-4acb-8e63-1cc9e8def897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:34.559831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 837/1675] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:34.572245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6c430-ac2b-4acb-8e63-1cc9e8def897 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:34.572245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6c430-ac2b-4acb-8e63-1cc9e8def897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:34.572245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6c430-ac2b-4acb-8e63-1cc9e8def897 admin admin] PATH_INFO: `/v3/projects/ded271760fdb466a8eb64de0192da786/users/a4a9184545684046876650f660c57b08/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:34.602158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6c430-ac2b-4acb-8e63-1cc9e8def897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded271760fdb466a8eb64de0192da786, user_id=a4a9184545684046876650f660c57b08, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:34.609486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6c430-ac2b-4acb-8e63-1cc9e8def897 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:34.652452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 839/1676] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:34 2022] PUT /identity/v3/projects/ded271760fdb466a8eb64de0192da786/users/a4a9184545684046876650f660c57b08/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:34.664532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b476e7-65c3-4a5e-963a-adb1d7ff82c3 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:34.664818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b476e7-65c3-4a5e-963a-adb1d7ff82c3 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:34.665096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b476e7-65c3-4a5e-963a-adb1d7ff82c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:34.716885 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c6b476e7-65c3-4a5e-963a-adb1d7ff82c3 None None] MFA Rules not processed for user `a4a9184545684046876650f660c57b08`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:34.718107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c6b476e7-65c3-4a5e-963a-adb1d7ff82c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:34.789913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6c430-ac2b-4acb-8e63-1cc9e8def897 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:34.792371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-bbd6c430-ac2b-4acb-8e63-1cc9e8def897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:34.855492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 838/1677] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:34.861575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b476e7-65c3-4a5e-963a-adb1d7ff82c3 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:35.001804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b476e7-65c3-4a5e-963a-adb1d7ff82c3 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:35.020595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcf3c7-b196-43a8-a35a-6e2439768004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:35.021662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcf3c7-b196-43a8-a35a-6e2439768004 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:35.023276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcf3c7-b196-43a8-a35a-6e2439768004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:35.024016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcf3c7-b196-43a8-a35a-6e2439768004 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:35.028854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcf3c7-b196-43a8-a35a-6e2439768004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:35.039455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcf3c7-b196-43a8-a35a-6e2439768004 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:35.047911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6c430-ac2b-4acb-8e63-1cc9e8def897 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:35.074095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0b4bf-921d-4b4c-bb2e-97c61f898b99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:35.074742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0b4bf-921d-4b4c-bb2e-97c61f898b99 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:35.075022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0b4bf-921d-4b4c-bb2e-97c61f898b99 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:35.075327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0b4bf-921d-4b4c-bb2e-97c61f898b99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:35.175190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 839/1678] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:34 2022] POST /identity/v3/projects => generated 365 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:16:35.178108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcf3c7-b196-43a8-a35a-6e2439768004 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:35.343658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0b4bf-921d-4b4c-bb2e-97c61f898b99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:35.351368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0b4bf-921d-4b4c-bb2e-97c61f898b99 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:35.387473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 840/1679] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:34 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:16:35.390495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:35.428465 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0b4bf-921d-4b4c-bb2e-97c61f898b99 service nova] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:35.431470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0ad0b4bf-921d-4b4c-bb2e-97c61f898b99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:35.510984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcf3c7-b196-43a8-a35a-6e2439768004 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:35.521080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0b4bf-921d-4b4c-bb2e-97c61f898b99 service nova] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:35.535383 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e65ee8-dd0c-4833-818a-0ce05a26566c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:35.539744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e65ee8-dd0c-4833-818a-0ce05a26566c admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:35.540046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e65ee8-dd0c-4833-818a-0ce05a26566c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:35.540323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e65ee8-dd0c-4833-818a-0ce05a26566c admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:35.541163 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e65ee8-dd0c-4833-818a-0ce05a26566c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:35.545528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:35.546588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:35.546873 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:35.547142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] PATH_INFO: `/v3/users/e3352d04e04942aa87180eccc1597d52` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:35.551576 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e65ee8-dd0c-4833-818a-0ce05a26566c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:35.552677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3352d04e04942aa87180eccc1597d52)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:35.565628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:35.587847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:35.638684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 840/1680] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:35 2022] POST /identity/v3/users => generated 289 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:35.644631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e65ee8-dd0c-4833-818a-0ce05a26566c admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:35.687053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e65ee8-dd0c-4833-818a-0ce05a26566c admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:35.706556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315eb05-19b0-44d3-8704-1f4103c53f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:35.719141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315eb05-19b0-44d3-8704-1f4103c53f3c admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:35.720522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315eb05-19b0-44d3-8704-1f4103c53f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:35.720826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315eb05-19b0-44d3-8704-1f4103c53f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:35.722654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315eb05-19b0-44d3-8704-1f4103c53f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:35.729045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315eb05-19b0-44d3-8704-1f4103c53f3c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:35.775514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 841/1681] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:35.780579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315eb05-19b0-44d3-8704-1f4103c53f3c admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:35.803270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315eb05-19b0-44d3-8704-1f4103c53f3c admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:35.808570 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3352d04e04942aa87180eccc1597d52'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:35.825816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35d783-7736-47eb-a638-cf86775cc1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:35.827611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35d783-7736-47eb-a638-cf86775cc1a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:35.828702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35d783-7736-47eb-a638-cf86775cc1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:35.830666 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35d783-7736-47eb-a638-cf86775cc1a9 admin admin] PATH_INFO: `/v3/projects/234df426c7e74cc89ce298959903bf2d/users/a36ca2f0fdd2437d8ba1870188ffa191/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:35.847862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3352d04e04942aa87180eccc1597d52'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:35.867272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3352d04e04942aa87180eccc1597d52'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:35.894146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3352d04e04942aa87180eccc1597d52'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:35.894814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3352d04e04942aa87180eccc1597d52'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:35.903469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35d783-7736-47eb-a638-cf86775cc1a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=234df426c7e74cc89ce298959903bf2d, user_id=a36ca2f0fdd2437d8ba1870188ffa191, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:35.910416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 841/1682] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:16:35 2022] DELETE /identity/v3/users/e3352d04e04942aa87180eccc1597d52 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:35.911480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:35.917812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:35.927707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35d783-7736-47eb-a638-cf86775cc1a9 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:35.977846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 842/1683] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:35 2022] PUT /identity/v3/projects/234df426c7e74cc89ce298959903bf2d/users/a36ca2f0fdd2437d8ba1870188ffa191/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:16:35.982624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156e3fe-552e-4386-97ad-a173cf3d75ec None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:35.983151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156e3fe-552e-4386-97ad-a173cf3d75ec None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:35.983642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156e3fe-552e-4386-97ad-a173cf3d75ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:36.062379 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-4156e3fe-552e-4386-97ad-a173cf3d75ec None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:36.070340 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4156e3fe-552e-4386-97ad-a173cf3d75ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:36.096722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed0c52-fb85-4e60-a149-f541dd9a2e73 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:36.231243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdd1d1-0ff8-4889-8a82-750575866b18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:36.232401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdd1d1-0ff8-4889-8a82-750575866b18 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:36.233054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdd1d1-0ff8-4889-8a82-750575866b18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:36.233697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdd1d1-0ff8-4889-8a82-750575866b18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:36.240233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-21cdd1d1-0ff8-4889-8a82-750575866b18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:36.311066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 843/1684] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:36.312316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156e3fe-552e-4386-97ad-a173cf3d75ec None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:36.317110 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4156e3fe-552e-4386-97ad-a173cf3d75ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:36.350854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156e3fe-552e-4386-97ad-a173cf3d75ec None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:36.367205 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694db980-33f0-4395-ab82-3c30befffe19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:36.367889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694db980-33f0-4395-ab82-3c30befffe19 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:36.368195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694db980-33f0-4395-ab82-3c30befffe19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:36.368984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694db980-33f0-4395-ab82-3c30befffe19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:36.369651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694db980-33f0-4395-ab82-3c30befffe19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:36.377011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694db980-33f0-4395-ab82-3c30befffe19 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:36.416549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 844/1685] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:36.417623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694db980-33f0-4395-ab82-3c30befffe19 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:36.445290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694db980-33f0-4395-ab82-3c30befffe19 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:36.452964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:36.455816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefdd0a-4de6-446c-af80-0429817f7926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:36.457684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefdd0a-4de6-446c-af80-0429817f7926 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:36.457962 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefdd0a-4de6-446c-af80-0429817f7926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:36.458235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefdd0a-4de6-446c-af80-0429817f7926 admin admin] PATH_INFO: `/v3/projects/234df426c7e74cc89ce298959903bf2d/users/a36ca2f0fdd2437d8ba1870188ffa191/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:36.472726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdd1d1-0ff8-4889-8a82-750575866b18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:36.476505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdd1d1-0ff8-4889-8a82-750575866b18 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:36.479002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefdd0a-4de6-446c-af80-0429817f7926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=234df426c7e74cc89ce298959903bf2d, user_id=a36ca2f0fdd2437d8ba1870188ffa191, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:36.484633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefdd0a-4de6-446c-af80-0429817f7926 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:36.512709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 842/1686] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:36.513769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdd1d1-0ff8-4889-8a82-750575866b18 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:36.519718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-21cdd1d1-0ff8-4889-8a82-750575866b18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:36.528795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 845/1687] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:36 2022] PUT /identity/v3/projects/234df426c7e74cc89ce298959903bf2d/users/a36ca2f0fdd2437d8ba1870188ffa191/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:36.530293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae37fb-a3e8-4608-a2ed-8644c038ae77 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:36.530580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae37fb-a3e8-4608-a2ed-8644c038ae77 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:36.530853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae37fb-a3e8-4608-a2ed-8644c038ae77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:36.630091 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-41ae37fb-a3e8-4608-a2ed-8644c038ae77 None None] MFA Rules not processed for user `a36ca2f0fdd2437d8ba1870188ffa191`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:36.637785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-41ae37fb-a3e8-4608-a2ed-8644c038ae77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:36.691296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdd1d1-0ff8-4889-8a82-750575866b18 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:36.833979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40985de5-05e1-4c0e-b1a8-7fc978e02cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:36.835408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40985de5-05e1-4c0e-b1a8-7fc978e02cad service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:36.836022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40985de5-05e1-4c0e-b1a8-7fc978e02cad service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:36.836877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40985de5-05e1-4c0e-b1a8-7fc978e02cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:36.855523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 846/1688] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:37.052100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40985de5-05e1-4c0e-b1a8-7fc978e02cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:37.056389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40985de5-05e1-4c0e-b1a8-7fc978e02cad service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:37.117283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 843/1689] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:37.348453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe153c3-050a-49fa-9cc1-743c76b012e3 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:37.348801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe153c3-050a-49fa-9cc1-743c76b012e3 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:37.349002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe153c3-050a-49fa-9cc1-743c76b012e3 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:37.350370 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 847/1690] 10.222.0.44 () {54 vars in 912 bytes} [Tue Aug 30 01:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:16:37.372408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4746ea-cf19-4d20-8e4c-452cc8ff8b68 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:37.372833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4746ea-cf19-4d20-8e4c-452cc8ff8b68 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:37.373224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4746ea-cf19-4d20-8e4c-452cc8ff8b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:37.443957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe153c3-050a-49fa-9cc1-743c76b012e3 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:37.446017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ffe153c3-050a-49fa-9cc1-743c76b012e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:37.456545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:37.551850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6f4746ea-cf19-4d20-8e4c-452cc8ff8b68 None None] MFA Rules not processed for user `86cea2f9238b42e0a7cb26bac494a8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:37.563768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6f4746ea-cf19-4d20-8e4c-452cc8ff8b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:37.590112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 844/1691] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:16:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:37.664713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4746ea-cf19-4d20-8e4c-452cc8ff8b68 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:37.668708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6f4746ea-cf19-4d20-8e4c-452cc8ff8b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:37.719288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe153c3-050a-49fa-9cc1-743c76b012e3 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:37.724900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4746ea-cf19-4d20-8e4c-452cc8ff8b68 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:37.736540 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0d2c9-1e73-420e-a29c-f088d31ea316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:37.737506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0d2c9-1e73-420e-a29c-f088d31ea316 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:37.737984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0d2c9-1e73-420e-a29c-f088d31ea316 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:37.738467 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0d2c9-1e73-420e-a29c-f088d31ea316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:37.749826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a6b92-e8d0-4387-9ef6-fa73a596574d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:37.750917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a6b92-e8d0-4387-9ef6-fa73a596574d service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:37.755347 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a6b92-e8d0-4387-9ef6-fa73a596574d service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:37.756054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a6b92-e8d0-4387-9ef6-fa73a596574d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:37.775577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0d2c9-1e73-420e-a29c-f088d31ea316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:37.786361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0d2c9-1e73-420e-a29c-f088d31ea316 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:37.799027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a6b92-e8d0-4387-9ef6-fa73a596574d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:37.804012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a6b92-e8d0-4387-9ef6-fa73a596574d service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:37.847053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 848/1692] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:16:37.851034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 845/1693] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:38.406698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a81e3f-91fc-46b5-9763-ad9c3bbe7e4a None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:38.407235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a81e3f-91fc-46b5-9763-ad9c3bbe7e4a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:38.407644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a81e3f-91fc-46b5-9763-ad9c3bbe7e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:38.489335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-85a81e3f-91fc-46b5-9763-ad9c3bbe7e4a None None] MFA Rules not processed for user `5b2e7c46b6aa4800834115196866b244`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:38.491327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-85a81e3f-91fc-46b5-9763-ad9c3bbe7e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:38.547558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a6b92-e8d0-4387-9ef6-fa73a596574d service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:38.566821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a6b92-e8d0-4387-9ef6-fa73a596574d service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:38.586815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112e821-97d5-410d-b182-5fdfd12b4811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:38.588951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112e821-97d5-410d-b182-5fdfd12b4811 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:38.589466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112e821-97d5-410d-b182-5fdfd12b4811 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:38.589972 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112e821-97d5-410d-b182-5fdfd12b4811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:38.594998 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5112e821-97d5-410d-b182-5fdfd12b4811 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:38.664990 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:38.729471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 849/1694] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:38.731794 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb032a-8b4c-402d-9bed-2006ab663945 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:38.732719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb032a-8b4c-402d-9bed-2006ab663945 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:38.733637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb032a-8b4c-402d-9bed-2006ab663945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:38.765889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-9ddb032a-8b4c-402d-9bed-2006ab663945 None None] MFA Rules not processed for user `5b2e7c46b6aa4800834115196866b244`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:38.767612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9ddb032a-8b4c-402d-9bed-2006ab663945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:38.771754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112e821-97d5-410d-b182-5fdfd12b4811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:38.774100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112e821-97d5-410d-b182-5fdfd12b4811 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:38.783750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 850/1695] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:38.789925 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb032a-8b4c-402d-9bed-2006ab663945 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:38.792417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9ddb032a-8b4c-402d-9bed-2006ab663945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:38.798397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 846/1696] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:39.130983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb032a-8b4c-402d-9bed-2006ab663945 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:39.165963 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112e821-97d5-410d-b182-5fdfd12b4811 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:39.169088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5112e821-97d5-410d-b182-5fdfd12b4811 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:39.176603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43019c-f7f6-45be-8aad-b8d9e6d0300d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:39.178481 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43019c-f7f6-45be-8aad-b8d9e6d0300d service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:39.179763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43019c-f7f6-45be-8aad-b8d9e6d0300d service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:39.181344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43019c-f7f6-45be-8aad-b8d9e6d0300d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:39.230824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43019c-f7f6-45be-8aad-b8d9e6d0300d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:39.233917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43019c-f7f6-45be-8aad-b8d9e6d0300d service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:39.259522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 851/1697] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:16:39.495419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112e821-97d5-410d-b182-5fdfd12b4811 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:39.520230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2f8e1-279d-4420-b489-f8a7b50d91ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08f34295f19b4d6ea405f1e8c8190db5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:39.521319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2f8e1-279d-4420-b489-f8a7b50d91ea service placement] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:39.521831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2f8e1-279d-4420-b489-f8a7b50d91ea service placement] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:39.522411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2f8e1-279d-4420-b489-f8a7b50d91ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:39.527497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-72f2f8e1-279d-4420-b489-f8a7b50d91ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:39.562133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2f8e1-279d-4420-b489-f8a7b50d91ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:39.569470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2f8e1-279d-4420-b489-f8a7b50d91ea service placement] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:39.615576 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d40d8-1e6f-4643-8931-44592210da59 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:39.616218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d40d8-1e6f-4643-8931-44592210da59 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:39.616672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d40d8-1e6f-4643-8931-44592210da59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:39.619145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 847/1698] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:16:39.767073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-663d40d8-1e6f-4643-8931-44592210da59 None None] MFA Rules not processed for user `6d5a0cf37e3b4d3084a178f83642c144`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:39.769216 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-663d40d8-1e6f-4643-8931-44592210da59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:39.778960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2f8e1-279d-4420-b489-f8a7b50d91ea service placement] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:39.780845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-72f2f8e1-279d-4420-b489-f8a7b50d91ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:39.821100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2f8e1-279d-4420-b489-f8a7b50d91ea service placement] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:39.826405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:39.844306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29604c93-7978-479a-96b4-674743d5bddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:39.849134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29604c93-7978-479a-96b4-674743d5bddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:39.849409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29604c93-7978-479a-96b4-674743d5bddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:39.849677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29604c93-7978-479a-96b4-674743d5bddb admin admin] PATH_INFO: `/v3/projects/c64e4eccf2c14dba96cbc1d8c4ab7831` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:39.856198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29604c93-7978-479a-96b4-674743d5bddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c64e4eccf2c14dba96cbc1d8c4ab7831)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:39.870920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29604c93-7978-479a-96b4-674743d5bddb admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:40.076075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 852/1699] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:40.199850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-29604c93-7978-479a-96b4-674743d5bddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c64e4eccf2c14dba96cbc1d8c4ab7831', 'request_id': 'req-29604c93-7978-479a-96b4-674743d5bddb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:40.229535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-29604c93-7978-479a-96b4-674743d5bddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c64e4eccf2c14dba96cbc1d8c4ab7831', 'request_id': 'req-29604c93-7978-479a-96b4-674743d5bddb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:40.241846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-29604c93-7978-479a-96b4-674743d5bddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c64e4eccf2c14dba96cbc1d8c4ab7831', 'request_id': 'req-29604c93-7978-479a-96b4-674743d5bddb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:40.302980 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-29604c93-7978-479a-96b4-674743d5bddb admin admin] The token cache is being invalidate because project c64e4eccf2c14dba96cbc1d8c4ab7831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:16:40.303342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-29604c93-7978-479a-96b4-674743d5bddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:40.311317 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-29604c93-7978-479a-96b4-674743d5bddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:40.315508 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 848/1700] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:16:39 2022] DELETE /identity/v3/projects/c64e4eccf2c14dba96cbc1d8c4ab7831 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:16:40.321837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d414e-b30a-41d7-af54-e1afc0db73f5 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:40.322286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d414e-b30a-41d7-af54-e1afc0db73f5 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:40.322734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d414e-b30a-41d7-af54-e1afc0db73f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:40.426708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-bf9d414e-b30a-41d7-af54-e1afc0db73f5 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:40.430772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-bf9d414e-b30a-41d7-af54-e1afc0db73f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:40.728405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 853/1701] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:40.735055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c617beb-bfb3-4ed3-ac9d-7cec02809644 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:40.735331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c617beb-bfb3-4ed3-ac9d-7cec02809644 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:40.735960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c617beb-bfb3-4ed3-ac9d-7cec02809644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:40.783425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-5c617beb-bfb3-4ed3-ac9d-7cec02809644 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:40.794537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5c617beb-bfb3-4ed3-ac9d-7cec02809644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:40.818079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 849/1702] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:40.820575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d414e-b30a-41d7-af54-e1afc0db73f5 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:40.838436 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d414e-b30a-41d7-af54-e1afc0db73f5 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:40.854590 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc116ba-69f1-4594-afbf-31022df6614e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:40.856022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc116ba-69f1-4594-afbf-31022df6614e admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:40.857776 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc116ba-69f1-4594-afbf-31022df6614e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:40.858534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc116ba-69f1-4594-afbf-31022df6614e admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:40.860078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc116ba-69f1-4594-afbf-31022df6614e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:40.860753 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc116ba-69f1-4594-afbf-31022df6614e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:40.865262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc116ba-69f1-4594-afbf-31022df6614e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:40.887502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 854/1703] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:16:40.893124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c617beb-bfb3-4ed3-ac9d-7cec02809644 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:40.906686 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c617beb-bfb3-4ed3-ac9d-7cec02809644 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:40.922512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00702ba0-e157-408c-aa92-f6e43eaa93ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:40.923226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00702ba0-e157-408c-aa92-f6e43eaa93ec admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:40.923507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00702ba0-e157-408c-aa92-f6e43eaa93ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:40.923947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00702ba0-e157-408c-aa92-f6e43eaa93ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:40.925040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00702ba0-e157-408c-aa92-f6e43eaa93ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:40.928537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00702ba0-e157-408c-aa92-f6e43eaa93ec admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:41.022944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 850/1704] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:40 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:16:41.024953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc116ba-69f1-4594-afbf-31022df6614e admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:41.042175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc116ba-69f1-4594-afbf-31022df6614e admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:41.311008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863750d-b979-4b40-9663-c0c0d360b18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:41.315378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863750d-b979-4b40-9663-c0c0d360b18e admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:41.329776 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863750d-b979-4b40-9663-c0c0d360b18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:41.330168 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863750d-b979-4b40-9663-c0c0d360b18e admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:41.330871 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863750d-b979-4b40-9663-c0c0d360b18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:41.336439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863750d-b979-4b40-9663-c0c0d360b18e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:41.411220 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 855/1705] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:41 2022] POST /identity/v3/users => generated 286 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:41.418863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00702ba0-e157-408c-aa92-f6e43eaa93ec admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:41.459965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00702ba0-e157-408c-aa92-f6e43eaa93ec admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:41.486286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23351c-ae77-40d3-af25-efdd52b7397a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:41.488181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23351c-ae77-40d3-af25-efdd52b7397a admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:41.489999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23351c-ae77-40d3-af25-efdd52b7397a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:41.490275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23351c-ae77-40d3-af25-efdd52b7397a admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:41.495701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23351c-ae77-40d3-af25-efdd52b7397a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:41.503520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23351c-ae77-40d3-af25-efdd52b7397a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:41.520978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:41.559790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 851/1706] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:41.560308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863750d-b979-4b40-9663-c0c0d360b18e admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:41.581004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863750d-b979-4b40-9663-c0c0d360b18e admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:41.612566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39daef-42d8-49a6-ac64-0648ae73f1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:41.613227 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39daef-42d8-49a6-ac64-0648ae73f1eb admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:41.613517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39daef-42d8-49a6-ac64-0648ae73f1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:41.613810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39daef-42d8-49a6-ac64-0648ae73f1eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:41.617013 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39daef-42d8-49a6-ac64-0648ae73f1eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:41.619790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39daef-42d8-49a6-ac64-0648ae73f1eb admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:41.658481 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 856/1707] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:16:41.663830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23351c-ae77-40d3-af25-efdd52b7397a admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:41.713400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23351c-ae77-40d3-af25-efdd52b7397a admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:41.733092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef898d0-c8e4-4736-9122-17220866cbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:41.734515 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef898d0-c8e4-4736-9122-17220866cbf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:41.735202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef898d0-c8e4-4736-9122-17220866cbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:41.735894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef898d0-c8e4-4736-9122-17220866cbf5 admin admin] PATH_INFO: `/v3/projects/b28386de5d164596b344b05f0d71848e/users/8ad9c402520043bd8373f29790564971/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:41.777933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515f70b-0c16-4d11-ab9c-8952affbb988 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:41.779024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515f70b-0c16-4d11-ab9c-8952affbb988 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:41.779559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515f70b-0c16-4d11-ab9c-8952affbb988 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:41.781210 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 857/1708] 10.222.0.44 () {54 vars in 912 bytes} [Tue Aug 30 01:16:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:16:41.787874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef898d0-c8e4-4736-9122-17220866cbf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28386de5d164596b344b05f0d71848e, user_id=8ad9c402520043bd8373f29790564971, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:41.801514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef898d0-c8e4-4736-9122-17220866cbf5 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:41.810605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f058d12-531d-439e-a093-13ea7640c424 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:41.810885 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f058d12-531d-439e-a093-13ea7640c424 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:41.811472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f058d12-531d-439e-a093-13ea7640c424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:41.851443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 852/1709] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:41 2022] PUT /identity/v3/projects/b28386de5d164596b344b05f0d71848e/users/8ad9c402520043bd8373f29790564971/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:41.864976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-6f058d12-531d-439e-a093-13ea7640c424 None None] MFA Rules not processed for user `86cea2f9238b42e0a7cb26bac494a8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:41.865844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a00a5-1dc1-41f9-ae02-04a257ff9165 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:41.865844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a00a5-1dc1-41f9-ae02-04a257ff9165 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:41.866092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a00a5-1dc1-41f9-ae02-04a257ff9165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:41.877493 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6f058d12-531d-439e-a093-13ea7640c424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:41.954133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-8b5a00a5-1dc1-41f9-ae02-04a257ff9165 None None] MFA Rules not processed for user `8ad9c402520043bd8373f29790564971`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:41.961755 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8b5a00a5-1dc1-41f9-ae02-04a257ff9165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:41.970305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:42.060331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 858/1710] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:16:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:42.100007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f058d12-531d-439e-a093-13ea7640c424 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:42.102000 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6f058d12-531d-439e-a093-13ea7640c424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:42.117263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f058d12-531d-439e-a093-13ea7640c424 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:42.144487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b188c-6fdf-4f0e-a50b-857dcb7ab8c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:42.146161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b188c-6fdf-4f0e-a50b-857dcb7ab8c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:42.146161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b188c-6fdf-4f0e-a50b-857dcb7ab8c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:42.157443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b188c-6fdf-4f0e-a50b-857dcb7ab8c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:42.180078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b188c-6fdf-4f0e-a50b-857dcb7ab8c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:42.198557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b188c-6fdf-4f0e-a50b-857dcb7ab8c1 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:42.235861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 859/1711] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:16:42.242478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3387f6-b73d-469a-81e3-76374d411bfb None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:42.244738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3387f6-b73d-469a-81e3-76374d411bfb None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:42.245129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3387f6-b73d-469a-81e3-76374d411bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:42.260007 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 853/1712] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:42.264332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a00a5-1dc1-41f9-ae02-04a257ff9165 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:42.300720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-bf3387f6-b73d-469a-81e3-76374d411bfb None None] MFA Rules not processed for user `d5bd0cb311db42d7882a4cf90b0d85e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:42.308148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-bf3387f6-b73d-469a-81e3-76374d411bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:42.465071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a00a5-1dc1-41f9-ae02-04a257ff9165 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:42.484247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425187c-beb0-4000-9ab2-b92ffe8446dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:42.487441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425187c-beb0-4000-9ab2-b92ffe8446dc admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:42.487737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425187c-beb0-4000-9ab2-b92ffe8446dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:42.488034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425187c-beb0-4000-9ab2-b92ffe8446dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:42.491085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425187c-beb0-4000-9ab2-b92ffe8446dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:42.493644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425187c-beb0-4000-9ab2-b92ffe8446dc admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:42.520600 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 860/1713] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:42.524635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3387f6-b73d-469a-81e3-76374d411bfb None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:42.530439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-bf3387f6-b73d-469a-81e3-76374d411bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:42.585237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3387f6-b73d-469a-81e3-76374d411bfb None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:42.586778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 854/1714] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:42 2022] POST /identity/v3/projects => generated 371 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:42.588228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425187c-beb0-4000-9ab2-b92ffe8446dc admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:42.680422 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:42.708775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425187c-beb0-4000-9ab2-b92ffe8446dc admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:42.729650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3796cb-be13-4363-84b6-d8a3059dc402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:42.730653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3796cb-be13-4363-84b6-d8a3059dc402 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:42.730653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3796cb-be13-4363-84b6-d8a3059dc402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:42.731255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3796cb-be13-4363-84b6-d8a3059dc402 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:42.732790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3796cb-be13-4363-84b6-d8a3059dc402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:42.736862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3796cb-be13-4363-84b6-d8a3059dc402 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:42.869735 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 855/1715] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:42 2022] POST /identity/v3/users => generated 292 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:42.873743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3796cb-be13-4363-84b6-d8a3059dc402 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:42.879198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9b3796cb-be13-4363-84b6-d8a3059dc402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:42.921605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:42.926289 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:42.928181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:42.928653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] PATH_INFO: `/v3/users/238d246c2cd64141a7be6fd56494d950` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:42.932487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] RBAC: Authorizing `identity:delete_user(user_id=238d246c2cd64141a7be6fd56494d950)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:42.939416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:42.965968 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3796cb-be13-4363-84b6-d8a3059dc402 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:42.980928 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:42.987967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:42.990404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:42.995477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] PATH_INFO: `/v3/users/a4a9184545684046876650f660c57b08` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:42.997177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a9184545684046876650f660c57b08)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:43.005066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:43.068792 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '238d246c2cd64141a7be6fd56494d950'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.085323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '238d246c2cd64141a7be6fd56494d950'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.086271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '238d246c2cd64141a7be6fd56494d950'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.087861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '238d246c2cd64141a7be6fd56494d950'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.092401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a9184545684046876650f660c57b08'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.116782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:43.121008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '238d246c2cd64141a7be6fd56494d950'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.125907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a9184545684046876650f660c57b08'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.154522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 861/1716] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:16:42 2022] DELETE /identity/v3/users/238d246c2cd64141a7be6fd56494d950 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:43.159183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:43.166775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a9184545684046876650f660c57b08'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.168084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:43.184427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a9184545684046876650f660c57b08'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.185455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a9184545684046876650f660c57b08'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.197934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 856/1717] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:16:42 2022] DELETE /identity/v3/users/a4a9184545684046876650f660c57b08 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:43.199554 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:43.204041 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:43.410847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e39f07-35e0-41ab-81ce-d40c7b0ee87d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:43.460240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51634a-8fc6-4382-a713-275a9cd067dc admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:43.478403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f4bd2-598f-4d29-80c0-605de324b813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:43.479064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f4bd2-598f-4d29-80c0-605de324b813 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:43.479306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f4bd2-598f-4d29-80c0-605de324b813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:43.479571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f4bd2-598f-4d29-80c0-605de324b813 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:43.480256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f4bd2-598f-4d29-80c0-605de324b813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:43.486101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f4bd2-598f-4d29-80c0-605de324b813 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:43.531722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 857/1718] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:43.533500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21375414-eaf4-474b-9607-0746d0fa162a None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:43.533829 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21375414-eaf4-474b-9607-0746d0fa162a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:43.534314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21375414-eaf4-474b-9607-0746d0fa162a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:43.573675 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-21375414-eaf4-474b-9607-0746d0fa162a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:43.577797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-21375414-eaf4-474b-9607-0746d0fa162a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:43.595656 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 858/1719] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:43.597159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d521bdd-bf1b-4153-9e6a-d32c54bcc957 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:43.597436 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d521bdd-bf1b-4153-9e6a-d32c54bcc957 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:43.597704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d521bdd-bf1b-4153-9e6a-d32c54bcc957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:43.641854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-3d521bdd-bf1b-4153-9e6a-d32c54bcc957 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:43.642192 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:43.642982 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3d521bdd-bf1b-4153-9e6a-d32c54bcc957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:43.655643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:43.658318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:43.658808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] PATH_INFO: `/v3/users/ae48c1e12a514e3c83b64204b7b4bc0d` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:43.661410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 859/1720] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:43.661882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae48c1e12a514e3c83b64204b7b4bc0d)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:43.662321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d521bdd-bf1b-4153-9e6a-d32c54bcc957 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:43.667210 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:43.682640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d521bdd-bf1b-4153-9e6a-d32c54bcc957 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:43.714559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af33fd-c0c3-4091-9372-b518d5a005e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:43.715205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af33fd-c0c3-4091-9372-b518d5a005e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:43.715539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af33fd-c0c3-4091-9372-b518d5a005e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:43.715861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af33fd-c0c3-4091-9372-b518d5a005e9 admin admin] PATH_INFO: `/v3/projects/32a9b1304bf94b0689359c20d1e14f20/users/ac7eceff6b864a208191b25497fe457e/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:43.777083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af33fd-c0c3-4091-9372-b518d5a005e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a9b1304bf94b0689359c20d1e14f20, user_id=ac7eceff6b864a208191b25497fe457e, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:43.780439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af33fd-c0c3-4091-9372-b518d5a005e9 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:43.796925 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae48c1e12a514e3c83b64204b7b4bc0d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.818403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 860/1721] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:43 2022] PUT /identity/v3/projects/32a9b1304bf94b0689359c20d1e14f20/users/ac7eceff6b864a208191b25497fe457e/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:43.818673 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae48c1e12a514e3c83b64204b7b4bc0d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.819489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af33fd-c0c3-4091-9372-b518d5a005e9 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:43.824727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae48c1e12a514e3c83b64204b7b4bc0d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.830759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-03af33fd-c0c3-4091-9372-b518d5a005e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:43.831089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae48c1e12a514e3c83b64204b7b4bc0d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.845222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae48c1e12a514e3c83b64204b7b4bc0d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.866492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:43.879001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 862/1722] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:16:43 2022] DELETE /identity/v3/users/ae48c1e12a514e3c83b64204b7b4bc0d => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:43.880539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:43.884465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:43.904233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:43.980229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5030f-155f-4906-9698-587591e5d686 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:43.985889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac9e37-6075-43bf-bdeb-fc9b24024ed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:43.986662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac9e37-6075-43bf-bdeb-fc9b24024ed6 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:43.987101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac9e37-6075-43bf-bdeb-fc9b24024ed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:43.987447 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac9e37-6075-43bf-bdeb-fc9b24024ed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.004616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-90ac9e37-6075-43bf-bdeb-fc9b24024ed6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:44.050287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af33fd-c0c3-4091-9372-b518d5a005e9 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:44.079989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74756ad4-5158-4e1b-8421-8d2a04931a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:44.080745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74756ad4-5158-4e1b-8421-8d2a04931a4f service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.081032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74756ad4-5158-4e1b-8421-8d2a04931a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.081306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74756ad4-5158-4e1b-8421-8d2a04931a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.090006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-74756ad4-5158-4e1b-8421-8d2a04931a4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:44.284037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac9e37-6075-43bf-bdeb-fc9b24024ed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:44.285747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac9e37-6075-43bf-bdeb-fc9b24024ed6 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:44.311840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 863/1723] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:44.312933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac9e37-6075-43bf-bdeb-fc9b24024ed6 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:44.314534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-90ac9e37-6075-43bf-bdeb-fc9b24024ed6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:44.330656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac9e37-6075-43bf-bdeb-fc9b24024ed6 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:44.338516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d1202-106c-4e28-bd7a-9426553871dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:44.340632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d1202-106c-4e28-bd7a-9426553871dd admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.341024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d1202-106c-4e28-bd7a-9426553871dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.343090 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d1202-106c-4e28-bd7a-9426553871dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.343617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d1202-106c-4e28-bd7a-9426553871dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:44.347746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d1202-106c-4e28-bd7a-9426553871dd admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:44.382013 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 864/1724] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:44.382432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60308bd2-b5a0-452d-89c5-caa8fb73d0d9 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.384800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60308bd2-b5a0-452d-89c5-caa8fb73d0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.384800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60308bd2-b5a0-452d-89c5-caa8fb73d0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.405752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74756ad4-5158-4e1b-8421-8d2a04931a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:44.407246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74756ad4-5158-4e1b-8421-8d2a04931a4f service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:44.410964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-60308bd2-b5a0-452d-89c5-caa8fb73d0d9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:44.412428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-60308bd2-b5a0-452d-89c5-caa8fb73d0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:44.437988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 865/1725] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:44.439186 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60308bd2-b5a0-452d-89c5-caa8fb73d0d9 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:44.450127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 861/1726] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:44.467376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74756ad4-5158-4e1b-8421-8d2a04931a4f service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:44.481722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74756ad4-5158-4e1b-8421-8d2a04931a4f service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:44.492230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3c2ac-6f90-4d74-b645-2d1450ce3cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:44.493554 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3c2ac-6f90-4d74-b645-2d1450ce3cfe service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.493768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3c2ac-6f90-4d74-b645-2d1450ce3cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.494118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3c2ac-6f90-4d74-b645-2d1450ce3cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.504940 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60308bd2-b5a0-452d-89c5-caa8fb73d0d9 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:44.519724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462534ef-ed11-4f90-b420-139b657aa41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:44.521145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462534ef-ed11-4f90-b420-139b657aa41d admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.523101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462534ef-ed11-4f90-b420-139b657aa41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.523101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462534ef-ed11-4f90-b420-139b657aa41d admin admin] PATH_INFO: `/v3/projects/32a9b1304bf94b0689359c20d1e14f20/users/ac7eceff6b864a208191b25497fe457e/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.532907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462534ef-ed11-4f90-b420-139b657aa41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a9b1304bf94b0689359c20d1e14f20, user_id=ac7eceff6b864a208191b25497fe457e, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:44.534525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3c2ac-6f90-4d74-b645-2d1450ce3cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:44.535980 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3c2ac-6f90-4d74-b645-2d1450ce3cfe service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:44.539203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462534ef-ed11-4f90-b420-139b657aa41d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:44.570385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 862/1727] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:16:44.579549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 866/1728] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:44 2022] PUT /identity/v3/projects/32a9b1304bf94b0689359c20d1e14f20/users/ac7eceff6b864a208191b25497fe457e/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:44.586494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694f867-e6ca-4342-ac3f-6a60130dd724 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.587027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694f867-e6ca-4342-ac3f-6a60130dd724 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.587466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694f867-e6ca-4342-ac3f-6a60130dd724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.592031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d3557-942a-4e74-ac55-33dd9334f8ca None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.593411 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d3557-942a-4e74-ac55-33dd9334f8ca None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.594856 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d3557-942a-4e74-ac55-33dd9334f8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.761929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d76d3557-942a-4e74-ac55-33dd9334f8ca None None] MFA Rules not processed for user `ac7eceff6b864a208191b25497fe457e`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:44.764623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d76d3557-942a-4e74-ac55-33dd9334f8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:44.823012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-f694f867-e6ca-4342-ac3f-6a60130dd724 None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:44.824471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f694f867-e6ca-4342-ac3f-6a60130dd724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:45.016713 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 867/1729] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:45.031051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d3557-942a-4e74-ac55-33dd9334f8ca None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:45.037072 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d76d3557-942a-4e74-ac55-33dd9334f8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:45.137582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 863/1730] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:16:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 552 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:45.195073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:45.433374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d3557-942a-4e74-ac55-33dd9334f8ca None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:45.453574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6999d9d-41af-4fd9-a86c-532a3e9674b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:45.454282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6999d9d-41af-4fd9-a86c-532a3e9674b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:45.454674 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6999d9d-41af-4fd9-a86c-532a3e9674b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:45.455070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6999d9d-41af-4fd9-a86c-532a3e9674b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:45.502436 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6999d9d-41af-4fd9-a86c-532a3e9674b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:45.504061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6999d9d-41af-4fd9-a86c-532a3e9674b7 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:45.514507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694f867-e6ca-4342-ac3f-6a60130dd724 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:45.516672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f694f867-e6ca-4342-ac3f-6a60130dd724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:45.535645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 868/1731] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:45.593558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:45.683794 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6999d9d-41af-4fd9-a86c-532a3e9674b7 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:45.701146 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6999d9d-41af-4fd9-a86c-532a3e9674b7 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:45.717625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943062ef-f377-4cba-a803-ab53ba5d61b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:45.718598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943062ef-f377-4cba-a803-ab53ba5d61b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:45.719089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943062ef-f377-4cba-a803-ab53ba5d61b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:45.719570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943062ef-f377-4cba-a803-ab53ba5d61b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:45.729739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-943062ef-f377-4cba-a803-ab53ba5d61b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:45.945247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694f867-e6ca-4342-ac3f-6a60130dd724 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:45.986243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d210b-aac2-47b1-aaf7-21cc39713246 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:45.988559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d210b-aac2-47b1-aaf7-21cc39713246 service swift] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:45.991105 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d210b-aac2-47b1-aaf7-21cc39713246 service swift] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:45.991505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d210b-aac2-47b1-aaf7-21cc39713246 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:46.028989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d210b-aac2-47b1-aaf7-21cc39713246 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:46.033420 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d210b-aac2-47b1-aaf7-21cc39713246 service swift] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:46.046551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943062ef-f377-4cba-a803-ab53ba5d61b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:46.049669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943062ef-f377-4cba-a803-ab53ba5d61b4 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:46.091151 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 864/1732] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:16:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 01:16:46.097972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 869/1733] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:46.279793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d210b-aac2-47b1-aaf7-21cc39713246 service swift] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:46.280462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b11d210b-aac2-47b1-aaf7-21cc39713246 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:46.672923 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d210b-aac2-47b1-aaf7-21cc39713246 service swift] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:46.688702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:46.689647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:46.693239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:46.693924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] PATH_INFO: `/v3/projects/ded271760fdb466a8eb64de0192da786` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:46.700322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ded271760fdb466a8eb64de0192da786)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:46.704912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:46.893562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ded271760fdb466a8eb64de0192da786', 'request_id': 'req-756c28fa-02ac-49ca-b244-64f8d92e146c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:46.969244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ded271760fdb466a8eb64de0192da786', 'request_id': 'req-756c28fa-02ac-49ca-b244-64f8d92e146c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:47.016426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ded271760fdb466a8eb64de0192da786', 'request_id': 'req-756c28fa-02ac-49ca-b244-64f8d92e146c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:47.064988 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] The token cache is being invalidate because project ded271760fdb466a8eb64de0192da786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:16:47.066967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:47.069763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:47.083331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 865/1734] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:16:46 2022] DELETE /identity/v3/projects/ded271760fdb466a8eb64de0192da786 => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:47.088915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943062ef-f377-4cba-a803-ab53ba5d61b4 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:47.115428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943062ef-f377-4cba-a803-ab53ba5d61b4 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:47.278799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:47.279917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:47.280510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:47.281134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] PATH_INFO: `/v3/users/a36ca2f0fdd2437d8ba1870188ffa191` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:47.283094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a36ca2f0fdd2437d8ba1870188ffa191)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:47.296711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:47.443355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a36ca2f0fdd2437d8ba1870188ffa191'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:47.464245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a36ca2f0fdd2437d8ba1870188ffa191'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:47.465724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a36ca2f0fdd2437d8ba1870188ffa191'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:47.467050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a36ca2f0fdd2437d8ba1870188ffa191'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:47.489991 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:47.518803 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a36ca2f0fdd2437d8ba1870188ffa191'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:47.569072 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 870/1735] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:16:47 2022] DELETE /identity/v3/users/a36ca2f0fdd2437d8ba1870188ffa191 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:47.717221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:47.720344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:47.763522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c28fa-02ac-49ca-b244-64f8d92e146c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:47.818212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:48.032225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:48.041488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:48.042321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:48.043037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] PATH_INFO: `/v3/projects/705008e3e864488f897ebbf513d569c0` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:48.048303 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=705008e3e864488f897ebbf513d569c0)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:48.058057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:48.286426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '705008e3e864488f897ebbf513d569c0', 'request_id': 'req-58d2c04e-c895-4d31-a663-7f3699e5ae8d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:48.327161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '705008e3e864488f897ebbf513d569c0', 'request_id': 'req-58d2c04e-c895-4d31-a663-7f3699e5ae8d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:48.382911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '705008e3e864488f897ebbf513d569c0', 'request_id': 'req-58d2c04e-c895-4d31-a663-7f3699e5ae8d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:48.460835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] The token cache is being invalidate because project 705008e3e864488f897ebbf513d569c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:16:48.461753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:48.470046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:48.478101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:48.479678 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:48.481987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 866/1736] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:16:47 2022] DELETE /identity/v3/projects/705008e3e864488f897ebbf513d569c0 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:48.491042 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:48.495819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:48.519196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e491a5-b08a-483d-a877-70ec8937ea51 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:48.791201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:48.792410 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:48.794831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:48.795491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] PATH_INFO: `/v3/projects/a0afbc954ebc4555bfc59a1899d9f4aa` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:48.800188 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0afbc954ebc4555bfc59a1899d9f4aa)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:48.803848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: WARNING py.warnings [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:16:48.803848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: warnings.warn(msg) Aug 30 01:16:48.803848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: Aug 30 01:16:48.808436 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:48.840992 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2c04e-c895-4d31-a663-7f3699e5ae8d admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:48.871407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2344b-f0db-4953-94b9-6b82c02a94d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:48.872470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2344b-f0db-4953-94b9-6b82c02a94d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:48.872921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2344b-f0db-4953-94b9-6b82c02a94d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:48.874313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2344b-f0db-4953-94b9-6b82c02a94d5 admin admin] PATH_INFO: `/v3/users/b59f3575f9d7401b9f91f0d09d4a13be` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:48.887916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2344b-f0db-4953-94b9-6b82c02a94d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b59f3575f9d7401b9f91f0d09d4a13be)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:48.892181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2344b-f0db-4953-94b9-6b82c02a94d5 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:49.134326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-46b2344b-f0db-4953-94b9-6b82c02a94d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b59f3575f9d7401b9f91f0d09d4a13be'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:49.147911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0afbc954ebc4555bfc59a1899d9f4aa', 'request_id': 'req-ab76f3dd-c416-4c42-9482-2286219df804'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:49.152992 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:49.214632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-46b2344b-f0db-4953-94b9-6b82c02a94d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b59f3575f9d7401b9f91f0d09d4a13be'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:49.250051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-46b2344b-f0db-4953-94b9-6b82c02a94d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b59f3575f9d7401b9f91f0d09d4a13be'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:49.293477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-46b2344b-f0db-4953-94b9-6b82c02a94d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b59f3575f9d7401b9f91f0d09d4a13be'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:49.299232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0afbc954ebc4555bfc59a1899d9f4aa', 'request_id': 'req-ab76f3dd-c416-4c42-9482-2286219df804'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:49.300720 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-46b2344b-f0db-4953-94b9-6b82c02a94d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b59f3575f9d7401b9f91f0d09d4a13be'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:49.332136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 867/1737] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:16:48 2022] DELETE /identity/v3/users/b59f3575f9d7401b9f91f0d09d4a13be => generated 0 bytes in 846 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:49.360446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0afbc954ebc4555bfc59a1899d9f4aa', 'request_id': 'req-ab76f3dd-c416-4c42-9482-2286219df804'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:49.454657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] The token cache is being invalidate because project a0afbc954ebc4555bfc59a1899d9f4aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:16:49.458178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:49.472397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ab76f3dd-c416-4c42-9482-2286219df804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:49.510258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 871/1738] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:16:48 2022] DELETE /identity/v3/projects/a0afbc954ebc4555bfc59a1899d9f4aa => generated 0 bytes in 1028 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:16:49.511483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b86dc3-7d3f-40ca-967c-683fd5cb3717 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:49.511791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b86dc3-7d3f-40ca-967c-683fd5cb3717 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:49.512130 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b86dc3-7d3f-40ca-967c-683fd5cb3717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:49.592732 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e4b86dc3-7d3f-40ca-967c-683fd5cb3717 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:49.595182 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e4b86dc3-7d3f-40ca-967c-683fd5cb3717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:49.806278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 868/1739] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:49.808514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943fbe0-44da-47c4-88ca-3a3cde0697b4 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:49.809365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943fbe0-44da-47c4-88ca-3a3cde0697b4 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:49.809794 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943fbe0-44da-47c4-88ca-3a3cde0697b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:49.921698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-5943fbe0-44da-47c4-88ca-3a3cde0697b4 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:49.923823 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5943fbe0-44da-47c4-88ca-3a3cde0697b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:49.962358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 872/1740] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:49.967165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b86dc3-7d3f-40ca-967c-683fd5cb3717 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:49.981137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b86dc3-7d3f-40ca-967c-683fd5cb3717 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:49.986148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943fbe0-44da-47c4-88ca-3a3cde0697b4 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:49.989907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5943fbe0-44da-47c4-88ca-3a3cde0697b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:50.015405 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1b3e7-3348-48ea-8816-fd22a5896bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:50.017594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1b3e7-3348-48ea-8816-fd22a5896bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:50.020859 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1b3e7-3348-48ea-8816-fd22a5896bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:50.021438 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1b3e7-3348-48ea-8816-fd22a5896bb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:50.024275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1b3e7-3348-48ea-8816-fd22a5896bb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:50.032185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1b3e7-3348-48ea-8816-fd22a5896bb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:50.049255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1b3e7-3348-48ea-8816-fd22a5896bb3 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:50.052430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943fbe0-44da-47c4-88ca-3a3cde0697b4 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:50.060902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:50.062212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:50.062752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:50.063472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f admin admin] PATH_INFO: `/v3/projects/234df426c7e74cc89ce298959903bf2d` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:50.065265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=234df426c7e74cc89ce298959903bf2d)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:50.069723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:50.148622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 869/1741] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:16:50.169495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1b3e7-3348-48ea-8816-fd22a5896bb3 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:50.199720 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1b3e7-3348-48ea-8816-fd22a5896bb3 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:50.225979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234df426c7e74cc89ce298959903bf2d', 'request_id': 'req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:50.245670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234df426c7e74cc89ce298959903bf2d', 'request_id': 'req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:50.261852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:50.277694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234df426c7e74cc89ce298959903bf2d', 'request_id': 'req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:50.325043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f admin admin] The token cache is being invalidate because project 234df426c7e74cc89ce298959903bf2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:16:50.325452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:50.326822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ccd24922-7ec0-41e7-abdb-2a6865e2b13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:50.330116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 873/1742] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:16:49 2022] DELETE /identity/v3/projects/234df426c7e74cc89ce298959903bf2d => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:50.336976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e2022-9b76-4421-91ef-58c9aeb84aa9 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:50.337975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e2022-9b76-4421-91ef-58c9aeb84aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:50.338955 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e2022-9b76-4421-91ef-58c9aeb84aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:50.417055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-1c1e2022-9b76-4421-91ef-58c9aeb84aa9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:50.418580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1c1e2022-9b76-4421-91ef-58c9aeb84aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:50.418817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ab971-9829-405a-a884-337cb35ae1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:50.419530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ab971-9829-405a-a884-337cb35ae1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:50.419812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ab971-9829-405a-a884-337cb35ae1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:50.421982 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ab971-9829-405a-a884-337cb35ae1d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:50.422192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ab971-9829-405a-a884-337cb35ae1d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:50.428350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ab971-9829-405a-a884-337cb35ae1d5 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:50.460745 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 874/1743] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:50.479068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961ff79-f384-448c-ae71-b87de6932fe8 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:50.479394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961ff79-f384-448c-ae71-b87de6932fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:50.479773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961ff79-f384-448c-ae71-b87de6932fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:50.531211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-3961ff79-f384-448c-ae71-b87de6932fe8 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:50.533694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3961ff79-f384-448c-ae71-b87de6932fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:50.536678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 870/1744] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:50 2022] POST /identity/v3/projects => generated 375 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:50.542951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f50e9d-3a88-439a-bcd9-3593a8b4a161 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:50.543375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f50e9d-3a88-439a-bcd9-3593a8b4a161 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:50.543656 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f50e9d-3a88-439a-bcd9-3593a8b4a161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:50.609547 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-33f50e9d-3a88-439a-bcd9-3593a8b4a161 None None] MFA Rules not processed for user `86cea2f9238b42e0a7cb26bac494a8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:50.610859 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-33f50e9d-3a88-439a-bcd9-3593a8b4a161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:50.716874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 875/1745] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:50.717832 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961ff79-f384-448c-ae71-b87de6932fe8 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:50.722409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3961ff79-f384-448c-ae71-b87de6932fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:50.747594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961ff79-f384-448c-ae71-b87de6932fe8 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:50.761705 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0c869-c9d5-422a-bb5d-650a236d6aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:50.762443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0c869-c9d5-422a-bb5d-650a236d6aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:50.762805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0c869-c9d5-422a-bb5d-650a236d6aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:50.763086 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0c869-c9d5-422a-bb5d-650a236d6aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:50.763747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0c869-c9d5-422a-bb5d-650a236d6aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:50.765818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0c869-c9d5-422a-bb5d-650a236d6aa0 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:50.785171 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:50.828955 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 876/1746] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:50 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:50.830006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0c869-c9d5-422a-bb5d-650a236d6aa0 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:50.856489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0c869-c9d5-422a-bb5d-650a236d6aa0 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:50.864429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 871/1747] 10.222.0.61 () {58 vars in 1015 bytes} [Tue Aug 30 01:16:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:50.866619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f50e9d-3a88-439a-bcd9-3593a8b4a161 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:50.887173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104919d-abba-4859-879a-a8b9e9574d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:50.891599 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104919d-abba-4859-879a-a8b9e9574d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:50.894227 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104919d-abba-4859-879a-a8b9e9574d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:50.897345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104919d-abba-4859-879a-a8b9e9574d3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:50.898285 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104919d-abba-4859-879a-a8b9e9574d3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:50.901373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f50e9d-3a88-439a-bcd9-3593a8b4a161 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:50.903077 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104919d-abba-4859-879a-a8b9e9574d3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:50.906190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104919d-abba-4859-879a-a8b9e9574d3d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:50.912846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b34559-732d-43f2-be6c-2eb89e0eb256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:50.914803 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b34559-732d-43f2-be6c-2eb89e0eb256 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:50.915136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b34559-732d-43f2-be6c-2eb89e0eb256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:50.915619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b34559-732d-43f2-be6c-2eb89e0eb256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:50.916404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b34559-732d-43f2-be6c-2eb89e0eb256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:50.922486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b34559-732d-43f2-be6c-2eb89e0eb256 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:50.948071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 872/1748] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:50.949395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b34559-732d-43f2-be6c-2eb89e0eb256 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:50.974522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b34559-732d-43f2-be6c-2eb89e0eb256 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:50.992288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 877/1749] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:50.993087 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104919d-abba-4859-879a-a8b9e9574d3d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:51.005582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc19c46-83fb-4703-be92-1ae36b66bcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:51.006213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19c46-83fb-4703-be92-1ae36b66bcdf admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:51.010532 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19c46-83fb-4703-be92-1ae36b66bcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:51.010896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19c46-83fb-4703-be92-1ae36b66bcdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:51.011559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc19c46-83fb-4703-be92-1ae36b66bcdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:51.018149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc19c46-83fb-4703-be92-1ae36b66bcdf admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:51.039041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104919d-abba-4859-879a-a8b9e9574d3d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:51.058487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 873/1750] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:51.063988 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc19c46-83fb-4703-be92-1ae36b66bcdf admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:51.080760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f737ebe-5daf-457f-ad40-9302781633a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:51.081346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f737ebe-5daf-457f-ad40-9302781633a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:51.081629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f737ebe-5daf-457f-ad40-9302781633a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:51.081905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f737ebe-5daf-457f-ad40-9302781633a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:51.082559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f737ebe-5daf-457f-ad40-9302781633a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:51.084541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f737ebe-5daf-457f-ad40-9302781633a3 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:51.097635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc19c46-83fb-4703-be92-1ae36b66bcdf admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:51.103198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07eff3-2f67-4f79-9294-278aeefc2dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:51.104039 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07eff3-2f67-4f79-9294-278aeefc2dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:51.104657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07eff3-2f67-4f79-9294-278aeefc2dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:51.105005 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07eff3-2f67-4f79-9294-278aeefc2dd4 admin admin] PATH_INFO: `/v3/projects/1d08fe83431444178f99d3d8e04bec87/users/0ab7330e3be540cdb3728b6ff2430631/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:51.144181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07eff3-2f67-4f79-9294-278aeefc2dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d08fe83431444178f99d3d8e04bec87, user_id=0ab7330e3be540cdb3728b6ff2430631, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:51.157135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07eff3-2f67-4f79-9294-278aeefc2dd4 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:51.208953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 874/1751] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:51 2022] PUT /identity/v3/projects/1d08fe83431444178f99d3d8e04bec87/users/0ab7330e3be540cdb3728b6ff2430631/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:16:51.216753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873c549-80ee-44bf-8d25-b7a6c4091726 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:51.217107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873c549-80ee-44bf-8d25-b7a6c4091726 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:51.218721 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873c549-80ee-44bf-8d25-b7a6c4091726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:51.232405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 878/1752] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:50 2022] POST /identity/v3/projects => generated 381 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:51.233657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f737ebe-5daf-457f-ad40-9302781633a3 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:51.359677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-f873c549-80ee-44bf-8d25-b7a6c4091726 None None] MFA Rules not processed for user `0ab7330e3be540cdb3728b6ff2430631`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:51.364033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f873c549-80ee-44bf-8d25-b7a6c4091726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:51.514453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f737ebe-5daf-457f-ad40-9302781633a3 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:51.534022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f72793-7391-4022-a8ca-95a6f6415d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:51.535076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f72793-7391-4022-a8ca-95a6f6415d20 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:51.535076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f72793-7391-4022-a8ca-95a6f6415d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:51.535339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f72793-7391-4022-a8ca-95a6f6415d20 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:51.537388 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f72793-7391-4022-a8ca-95a6f6415d20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:51.538488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f72793-7391-4022-a8ca-95a6f6415d20 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:51.616225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 879/1753] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:51 2022] POST /identity/v3/users => generated 291 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:51.617184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f72793-7391-4022-a8ca-95a6f6415d20 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:51.630324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-17f72793-7391-4022-a8ca-95a6f6415d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:51.664774 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:DELETE=3,keystone:SELECT=89,keystone:INSERT=8 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:51.718536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 875/1754] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:51.720485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873c549-80ee-44bf-8d25-b7a6c4091726 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:51.737691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873c549-80ee-44bf-8d25-b7a6c4091726 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:51.745467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe92b8-ec57-4add-927e-a8b34a62cee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:51.746374 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe92b8-ec57-4add-927e-a8b34a62cee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:51.747048 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe92b8-ec57-4add-927e-a8b34a62cee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:51.747492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe92b8-ec57-4add-927e-a8b34a62cee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:51.748357 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe92b8-ec57-4add-927e-a8b34a62cee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:51.751241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe92b8-ec57-4add-927e-a8b34a62cee8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:51.770453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 876/1755] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:51.772058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe92b8-ec57-4add-927e-a8b34a62cee8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:51.778948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1efe92b8-ec57-4add-927e-a8b34a62cee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:51.782683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f72793-7391-4022-a8ca-95a6f6415d20 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:51.795506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3cf76-264d-4330-96bf-cd89375a8e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:51.797439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3cf76-264d-4330-96bf-cd89375a8e9e service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:51.799920 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3cf76-264d-4330-96bf-cd89375a8e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:51.800752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3cf76-264d-4330-96bf-cd89375a8e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:51.854507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3cf76-264d-4330-96bf-cd89375a8e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:51.854507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3cf76-264d-4330-96bf-cd89375a8e9e service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:51.913108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 880/1756] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:51.917232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3cf76-264d-4330-96bf-cd89375a8e9e service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:52.001177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3cf76-264d-4330-96bf-cd89375a8e9e service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:52.014072 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a00cd-7e04-4314-99f5-b6f4c9d7841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:52.017554 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a00cd-7e04-4314-99f5-b6f4c9d7841d admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:52.017964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a00cd-7e04-4314-99f5-b6f4c9d7841d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:52.019224 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a00cd-7e04-4314-99f5-b6f4c9d7841d admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:52.020027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a00cd-7e04-4314-99f5-b6f4c9d7841d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:52.023017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a00cd-7e04-4314-99f5-b6f4c9d7841d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:52.098260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 881/1757] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:52.100563 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a00cd-7e04-4314-99f5-b6f4c9d7841d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:52.138376 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a00cd-7e04-4314-99f5-b6f4c9d7841d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:52.161749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490332ab-5085-487d-9bd9-c844e3014850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:52.162456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490332ab-5085-487d-9bd9-c844e3014850 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:52.162728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490332ab-5085-487d-9bd9-c844e3014850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:52.163008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490332ab-5085-487d-9bd9-c844e3014850 admin admin] PATH_INFO: `/v3/projects/89ff3113974a442c899ae9a54df289eb/users/3544aca3cd8a4a93b6648fb9b24cfe44/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:52.175411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe92b8-ec57-4add-927e-a8b34a62cee8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:52.214374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490332ab-5085-487d-9bd9-c844e3014850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ff3113974a442c899ae9a54df289eb, user_id=3544aca3cd8a4a93b6648fb9b24cfe44, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:52.217925 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490332ab-5085-487d-9bd9-c844e3014850 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:52.224267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c23e70-c6d9-4f69-81b1-3d928d01d4e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:52.230649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c23e70-c6d9-4f69-81b1-3d928d01d4e5 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:52.232524 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c23e70-c6d9-4f69-81b1-3d928d01d4e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:52.232987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c23e70-c6d9-4f69-81b1-3d928d01d4e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:52.248391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 882/1758] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:52 2022] PUT /identity/v3/projects/89ff3113974a442c899ae9a54df289eb/users/3544aca3cd8a4a93b6648fb9b24cfe44/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:52.250144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b7f73-b916-4861-98c8-50ce1d5776d1 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:52.251519 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b7f73-b916-4861-98c8-50ce1d5776d1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:52.251519 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b7f73-b916-4861-98c8-50ce1d5776d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:52.268627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c23e70-c6d9-4f69-81b1-3d928d01d4e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:52.275280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c23e70-c6d9-4f69-81b1-3d928d01d4e5 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:52.301697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 877/1759] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:52.330584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d81b7f73-b916-4861-98c8-50ce1d5776d1 None None] MFA Rules not processed for user `3544aca3cd8a4a93b6648fb9b24cfe44`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:52.332098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d81b7f73-b916-4861-98c8-50ce1d5776d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:52.511196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 883/1760] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:52.519195 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c23e70-c6d9-4f69-81b1-3d928d01d4e5 service nova] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:52.547068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c23e70-c6d9-4f69-81b1-3d928d01d4e5 service nova] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:52.552188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:52.703579 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8723b-5c85-49d2-83b3-c7967d412f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:52.704262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8723b-5c85-49d2-83b3-c7967d412f0a admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:52.704717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8723b-5c85-49d2-83b3-c7967d412f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:52.704811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8723b-5c85-49d2-83b3-c7967d412f0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:52.717023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8723b-5c85-49d2-83b3-c7967d412f0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:52.719072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8723b-5c85-49d2-83b3-c7967d412f0a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:52.786540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 878/1761] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:52 2022] POST /identity/v3/projects => generated 379 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:52.790798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b7f73-b916-4861-98c8-50ce1d5776d1 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:52.807104 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b7f73-b916-4861-98c8-50ce1d5776d1 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:52.835192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8723b-5c85-49d2-83b3-c7967d412f0a admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:52.839922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-30a8723b-5c85-49d2-83b3-c7967d412f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:53.139595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a90f6-45af-42d7-a402-d07542399a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:53.143562 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a90f6-45af-42d7-a402-d07542399a03 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:53.144351 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a90f6-45af-42d7-a402-d07542399a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:53.144628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a90f6-45af-42d7-a402-d07542399a03 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:53.145305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a90f6-45af-42d7-a402-d07542399a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:53.147309 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a90f6-45af-42d7-a402-d07542399a03 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:53.155478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8723b-5c85-49d2-83b3-c7967d412f0a admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:53.175981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:53.176683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:53.176966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:53.177242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] PATH_INFO: `/v3/projects/75ad412645e24b9dbf7f38c6571108fd` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:53.179920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75ad412645e24b9dbf7f38c6571108fd)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:53.183791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:53.197238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:53.216888 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 884/1762] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:52 2022] POST /identity/v3/users => generated 296 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:53.219269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a90f6-45af-42d7-a402-d07542399a03 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:53.258598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a90f6-45af-42d7-a402-d07542399a03 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:53.264731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:53.265637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:53.266415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:53.266911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] PATH_INFO: `/v3/users/0ab7330e3be540cdb3728b6ff2430631` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:53.268648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ab7330e3be540cdb3728b6ff2430631)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:53.272251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:53.329858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75ad412645e24b9dbf7f38c6571108fd', 'request_id': 'req-72839e13-1b18-46d6-be2a-77ac13510060'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.359640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75ad412645e24b9dbf7f38c6571108fd', 'request_id': 'req-72839e13-1b18-46d6-be2a-77ac13510060'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.368231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75ad412645e24b9dbf7f38c6571108fd', 'request_id': 'req-72839e13-1b18-46d6-be2a-77ac13510060'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.387309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] The token cache is being invalidate because project 75ad412645e24b9dbf7f38c6571108fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:16:53.387602 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.388445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.396576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 879/1763] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:16:52 2022] DELETE /identity/v3/projects/75ad412645e24b9dbf7f38c6571108fd => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:53.398934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:53.400719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:53.411604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ab7330e3be540cdb3728b6ff2430631'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.440632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:53.441665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ab7330e3be540cdb3728b6ff2430631'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.444477 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ab7330e3be540cdb3728b6ff2430631'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.452255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ab7330e3be540cdb3728b6ff2430631'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.488308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ab7330e3be540cdb3728b6ff2430631'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.509040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 885/1764] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:16:53 2022] DELETE /identity/v3/users/0ab7330e3be540cdb3728b6ff2430631 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:53.510426 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:53.512527 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:53.537533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72839e13-1b18-46d6-be2a-77ac13510060 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:53.680214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa1ba7-0cd9-497b-9fee-b073ed66745a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:53.681155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa1ba7-0cd9-497b-9fee-b073ed66745a admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:53.681439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa1ba7-0cd9-497b-9fee-b073ed66745a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:53.681758 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa1ba7-0cd9-497b-9fee-b073ed66745a admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:53.682820 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa1ba7-0cd9-497b-9fee-b073ed66745a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:53.691503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa1ba7-0cd9-497b-9fee-b073ed66745a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:53.717620 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a51acf-f208-4891-99c7-fee9f3e6aceb admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:53.718331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 880/1765] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:53.720448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f86d6-b61c-42b6-9d85-d090e19c0bef None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:53.721233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f86d6-b61c-42b6-9d85-d090e19c0bef None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:53.721495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f86d6-b61c-42b6-9d85-d090e19c0bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:53.731108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:53.731786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:53.732097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:53.732364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] PATH_INFO: `/v3/users/77eafa25e3f542aebe2df9eed440888b` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:53.734431 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77eafa25e3f542aebe2df9eed440888b)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:53.739566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:53.773190 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7c2f86d6-b61c-42b6-9d85-d090e19c0bef None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:53.777532 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7c2f86d6-b61c-42b6-9d85-d090e19c0bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:53.810642 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 881/1766] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:53.810642 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f86d6-b61c-42b6-9d85-d090e19c0bef None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:53.816238 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7c2f86d6-b61c-42b6-9d85-d090e19c0bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:53.861876 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f86d6-b61c-42b6-9d85-d090e19c0bef None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:53.874679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77eafa25e3f542aebe2df9eed440888b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.876606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dec768-c9d6-4008-b3d0-0c80170f905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:53.877290 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dec768-c9d6-4008-b3d0-0c80170f905b admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:53.877569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dec768-c9d6-4008-b3d0-0c80170f905b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:53.877843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dec768-c9d6-4008-b3d0-0c80170f905b admin admin] PATH_INFO: `/v3/projects/8ae145739a534e75a045d2ffdd69779c/users/0a9c720d5ce6484a8cea554d71f8985d/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:53.898345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77eafa25e3f542aebe2df9eed440888b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.899247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77eafa25e3f542aebe2df9eed440888b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.899956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77eafa25e3f542aebe2df9eed440888b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.924938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dec768-c9d6-4008-b3d0-0c80170f905b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae145739a534e75a045d2ffdd69779c, user_id=0a9c720d5ce6484a8cea554d71f8985d, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:53.929850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77eafa25e3f542aebe2df9eed440888b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:53.932067 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dec768-c9d6-4008-b3d0-0c80170f905b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:53.937629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:53.960321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 882/1767] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:53 2022] PUT /identity/v3/projects/8ae145739a534e75a045d2ffdd69779c/users/0a9c720d5ce6484a8cea554d71f8985d/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:53.962167 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dec768-c9d6-4008-b3d0-0c80170f905b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:53.967024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a1dec768-c9d6-4008-b3d0-0c80170f905b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:53.974161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 886/1768] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:16:53 2022] DELETE /identity/v3/users/77eafa25e3f542aebe2df9eed440888b => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:53.975540 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:53.978826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:54.122037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dec768-c9d6-4008-b3d0-0c80170f905b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:54.161249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd6a9b-b650-4942-b3fd-b3ccbb447677 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:54.176857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5df7b5-4a2a-42b3-a143-31fbb60220de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:54.177830 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5df7b5-4a2a-42b3-a143-31fbb60220de admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:54.178264 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5df7b5-4a2a-42b3-a143-31fbb60220de admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:54.178695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5df7b5-4a2a-42b3-a143-31fbb60220de admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:54.179403 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5df7b5-4a2a-42b3-a143-31fbb60220de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:54.181883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5df7b5-4a2a-42b3-a143-31fbb60220de admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:54.207875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 887/1769] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:54.210585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5df7b5-4a2a-42b3-a143-31fbb60220de admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:54.227984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5df7b5-4a2a-42b3-a143-31fbb60220de admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:54.244271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01023137-567e-4ba4-949b-b2685d8f7f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:54.246719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01023137-567e-4ba4-949b-b2685d8f7f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:54.247793 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01023137-567e-4ba4-949b-b2685d8f7f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:54.248095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01023137-567e-4ba4-949b-b2685d8f7f1d admin admin] PATH_INFO: `/v3/projects/8ae145739a534e75a045d2ffdd69779c/users/0a9c720d5ce6484a8cea554d71f8985d/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:54.255138 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01023137-567e-4ba4-949b-b2685d8f7f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae145739a534e75a045d2ffdd69779c, user_id=0a9c720d5ce6484a8cea554d71f8985d, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:54.258413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc60f9-bdf5-40a4-aaf5-90ed142b5677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:54.260341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc60f9-bdf5-40a4-aaf5-90ed142b5677 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:54.262266 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc60f9-bdf5-40a4-aaf5-90ed142b5677 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:54.263291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01023137-567e-4ba4-949b-b2685d8f7f1d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:54.263427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc60f9-bdf5-40a4-aaf5-90ed142b5677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:54.267951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-08bc60f9-bdf5-40a4-aaf5-90ed142b5677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:54.295490 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 888/1770] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:54 2022] PUT /identity/v3/projects/8ae145739a534e75a045d2ffdd69779c/users/0a9c720d5ce6484a8cea554d71f8985d/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:54.300292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:54.310495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebd656-bedc-4334-a948-8f40077f02a5 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:54.310979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebd656-bedc-4334-a948-8f40077f02a5 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:54.311345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebd656-bedc-4334-a948-8f40077f02a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:54.312171 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc60f9-bdf5-40a4-aaf5-90ed142b5677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:54.313659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc60f9-bdf5-40a4-aaf5-90ed142b5677 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:54.344341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 883/1771] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:54.396727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-89ebd656-bedc-4334-a948-8f40077f02a5 None None] MFA Rules not processed for user `0a9c720d5ce6484a8cea554d71f8985d`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:54.402739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-89ebd656-bedc-4334-a948-8f40077f02a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:54.485875 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dc2a8-6c96-41e5-8db9-1512fb3fbd26 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:54.487134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dc2a8-6c96-41e5-8db9-1512fb3fbd26 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:54.487562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dc2a8-6c96-41e5-8db9-1512fb3fbd26 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:54.498860 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 884/1772] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:16:54 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:16:54.560641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c311d23-386d-4ecb-93e6-c814f972bcbd None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:54.561091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c311d23-386d-4ecb-93e6-c814f972bcbd None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:54.562106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c311d23-386d-4ecb-93e6-c814f972bcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:54.643008 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-3c311d23-386d-4ecb-93e6-c814f972bcbd None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:54.647541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3c311d23-386d-4ecb-93e6-c814f972bcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:54.668626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 889/1773] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:54.675359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebd656-bedc-4334-a948-8f40077f02a5 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:54.675359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-89ebd656-bedc-4334-a948-8f40077f02a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:54.881342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebd656-bedc-4334-a948-8f40077f02a5 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:54.899962 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:54.902558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:54.903671 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:54.904459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f admin admin] PATH_INFO: `/v3/projects/6f5127a35f8c472fbdac4f5c0d784029` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:54.909326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f5127a35f8c472fbdac4f5c0d784029)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:54.917614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:54.922866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:54.993366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 885/1774] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:16:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:54.998059 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c311d23-386d-4ecb-93e6-c814f972bcbd None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:55.003160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3c311d23-386d-4ecb-93e6-c814f972bcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:55.131538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f5127a35f8c472fbdac4f5c0d784029', 'request_id': 'req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:55.154456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f5127a35f8c472fbdac4f5c0d784029', 'request_id': 'req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:55.169090 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f5127a35f8c472fbdac4f5c0d784029', 'request_id': 'req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:55.202416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f admin admin] The token cache is being invalidate because project 6f5127a35f8c472fbdac4f5c0d784029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:16:55.202734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:55.205903 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f90216d6-a39a-4dd0-b8f7-68ff0493cc8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:55.207918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 890/1775] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:16:54 2022] DELETE /identity/v3/projects/6f5127a35f8c472fbdac4f5c0d784029 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:55.215421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52a71e-e31c-4efe-825f-a63f96edc0eb None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:55.215708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52a71e-e31c-4efe-825f-a63f96edc0eb None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:55.216014 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52a71e-e31c-4efe-825f-a63f96edc0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:55.263480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c311d23-386d-4ecb-93e6-c814f972bcbd None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:55.270400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-4f52a71e-e31c-4efe-825f-a63f96edc0eb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:55.276976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4f52a71e-e31c-4efe-825f-a63f96edc0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:55.448828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124430e6-ccae-41dc-8434-1fbbcbe13de5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:55.450379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124430e6-ccae-41dc-8434-1fbbcbe13de5 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:55.450808 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124430e6-ccae-41dc-8434-1fbbcbe13de5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:55.451222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124430e6-ccae-41dc-8434-1fbbcbe13de5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:55.453996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-124430e6-ccae-41dc-8434-1fbbcbe13de5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:55.480126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 891/1776] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:55.482430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52a71e-e31c-4efe-825f-a63f96edc0eb None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:55.485588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4f52a71e-e31c-4efe-825f-a63f96edc0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:55.640614 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:55.672080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124430e6-ccae-41dc-8434-1fbbcbe13de5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:55.676849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124430e6-ccae-41dc-8434-1fbbcbe13de5 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:55.720855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 886/1777] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:55.725591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52a71e-e31c-4efe-825f-a63f96edc0eb None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:55.725835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47854c4a-09d2-4f1e-a1f3-4c5c905fcf7b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:55.725835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47854c4a-09d2-4f1e-a1f3-4c5c905fcf7b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:55.730396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47854c4a-09d2-4f1e-a1f3-4c5c905fcf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:55.746504 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35b602-8f09-4296-994e-e4da551b6348 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:55.746917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35b602-8f09-4296-994e-e4da551b6348 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:55.747249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35b602-8f09-4296-994e-e4da551b6348 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:55.747533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35b602-8f09-4296-994e-e4da551b6348 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:55.749669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9b35b602-8f09-4296-994e-e4da551b6348 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:55.777640 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35b602-8f09-4296-994e-e4da551b6348 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:55.779447 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35b602-8f09-4296-994e-e4da551b6348 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:55.816414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 892/1778] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:55.831206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-47854c4a-09d2-4f1e-a1f3-4c5c905fcf7b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:55.837918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-47854c4a-09d2-4f1e-a1f3-4c5c905fcf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:55.880999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 887/1779] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:55.885656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35b602-8f09-4296-994e-e4da551b6348 service nova] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:55.909945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35b602-8f09-4296-994e-e4da551b6348 service nova] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:55.928257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b2e81-3ac6-40a4-be9a-ff995408c4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:55.930525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b2e81-3ac6-40a4-be9a-ff995408c4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:55.930819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b2e81-3ac6-40a4-be9a-ff995408c4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:55.931094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b2e81-3ac6-40a4-be9a-ff995408c4c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:55.931761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b2e81-3ac6-40a4-be9a-ff995408c4c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:55.932093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b2e81-3ac6-40a4-be9a-ff995408c4c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:55.936654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b2e81-3ac6-40a4-be9a-ff995408c4c1 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:55.990181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47854c4a-09d2-4f1e-a1f3-4c5c905fcf7b None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:56.004606 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 893/1780] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:56.006218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47854c4a-09d2-4f1e-a1f3-4c5c905fcf7b None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:56.034613 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aea226-9357-4707-90dc-e54a2ab67197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:56.035263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aea226-9357-4707-90dc-e54a2ab67197 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:56.035578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aea226-9357-4707-90dc-e54a2ab67197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:56.035851 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aea226-9357-4707-90dc-e54a2ab67197 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:56.038832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aea226-9357-4707-90dc-e54a2ab67197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:56.043014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aea226-9357-4707-90dc-e54a2ab67197 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:56.138562 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b2e81-3ac6-40a4-be9a-ff995408c4c1 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:56.144896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:56.152496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 888/1781] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:55 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:16:56.164478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b2e81-3ac6-40a4-be9a-ff995408c4c1 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:56.319544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c4117-2329-4054-aa69-49949cb0a4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:56.321071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c4117-2329-4054-aa69-49949cb0a4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:56.321071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c4117-2329-4054-aa69-49949cb0a4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:56.322221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c4117-2329-4054-aa69-49949cb0a4a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:56.322901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c4117-2329-4054-aa69-49949cb0a4a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:56.325102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c4117-2329-4054-aa69-49949cb0a4a9 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:56.366735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 894/1782] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:56 2022] POST /identity/v3/users => generated 292 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:16:56.371453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aea226-9357-4707-90dc-e54a2ab67197 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:56.386315 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aea226-9357-4707-90dc-e54a2ab67197 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:56.400799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9af73f-f12c-45df-bf1d-8d060cd67127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:56.401589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9af73f-f12c-45df-bf1d-8d060cd67127 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:56.402016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9af73f-f12c-45df-bf1d-8d060cd67127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:56.402188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9af73f-f12c-45df-bf1d-8d060cd67127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:56.402805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9af73f-f12c-45df-bf1d-8d060cd67127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:56.404487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9af73f-f12c-45df-bf1d-8d060cd67127 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:56.416061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c4117-2329-4054-aa69-49949cb0a4a9 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:56.417910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e26c4117-2329-4054-aa69-49949cb0a4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:56.446787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 889/1783] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:56.451315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c4117-2329-4054-aa69-49949cb0a4a9 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:56.462320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9af73f-f12c-45df-bf1d-8d060cd67127 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:56.479867 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537f236-8ca5-471f-b713-238595ab4a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:56.480582 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537f236-8ca5-471f-b713-238595ab4a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:56.480863 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537f236-8ca5-471f-b713-238595ab4a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:56.481144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537f236-8ca5-471f-b713-238595ab4a4e admin admin] PATH_INFO: `/v3/projects/1d08fe83431444178f99d3d8e04bec87` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:56.482814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537f236-8ca5-471f-b713-238595ab4a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d08fe83431444178f99d3d8e04bec87)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:56.484930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537f236-8ca5-471f-b713-238595ab4a4e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:56.491129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9af73f-f12c-45df-bf1d-8d060cd67127 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:56.515478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261c39d-f00c-4106-b695-2d0cb795b6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:56.516309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261c39d-f00c-4106-b695-2d0cb795b6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:56.516607 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261c39d-f00c-4106-b695-2d0cb795b6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:56.516953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261c39d-f00c-4106-b695-2d0cb795b6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:56.517519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261c39d-f00c-4106-b695-2d0cb795b6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:56.521871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261c39d-f00c-4106-b695-2d0cb795b6d2 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:56.568338 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 890/1784] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:56.580977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261c39d-f00c-4106-b695-2d0cb795b6d2 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:56.611449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6537f236-8ca5-471f-b713-238595ab4a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d08fe83431444178f99d3d8e04bec87', 'request_id': 'req-6537f236-8ca5-471f-b713-238595ab4a4e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:56.626307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6537f236-8ca5-471f-b713-238595ab4a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d08fe83431444178f99d3d8e04bec87', 'request_id': 'req-6537f236-8ca5-471f-b713-238595ab4a4e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:56.639149 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6537f236-8ca5-471f-b713-238595ab4a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d08fe83431444178f99d3d8e04bec87', 'request_id': 'req-6537f236-8ca5-471f-b713-238595ab4a4e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:56.660257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6537f236-8ca5-471f-b713-238595ab4a4e admin admin] The token cache is being invalidate because project 1d08fe83431444178f99d3d8e04bec87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:16:56.660550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6537f236-8ca5-471f-b713-238595ab4a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:56.661806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6537f236-8ca5-471f-b713-238595ab4a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:56.669707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 895/1785] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:16:56 2022] DELETE /identity/v3/projects/1d08fe83431444178f99d3d8e04bec87 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:56.669944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261c39d-f00c-4106-b695-2d0cb795b6d2 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:56.674934 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03deac-ea01-473e-9aff-902facd3f3e8 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:56.675363 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03deac-ea01-473e-9aff-902facd3f3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:56.676117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03deac-ea01-473e-9aff-902facd3f3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:56.767038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-4e03deac-ea01-473e-9aff-902facd3f3e8 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:56.771534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4e03deac-ea01-473e-9aff-902facd3f3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:56.960273 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8943625-0982-421b-aee5-840906bf6e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:56.965219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8943625-0982-421b-aee5-840906bf6e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:56.965810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8943625-0982-421b-aee5-840906bf6e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:56.967497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8943625-0982-421b-aee5-840906bf6e85 admin admin] PATH_INFO: `/v3/projects/d5e393097580423d841061b0f54ef868/users/d5c0292656124047bb3bcf7dbf4eb55a/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:57.021821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8943625-0982-421b-aee5-840906bf6e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e393097580423d841061b0f54ef868, user_id=d5c0292656124047bb3bcf7dbf4eb55a, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:57.028956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8943625-0982-421b-aee5-840906bf6e85 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:57.073675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 896/1786] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:57.088184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab3c2c-2199-4639-acb6-6f96c00462ff None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:57.088620 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab3c2c-2199-4639-acb6-6f96c00462ff None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:57.088900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab3c2c-2199-4639-acb6-6f96c00462ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:57.121034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 891/1787] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:56 2022] PUT /identity/v3/projects/d5e393097580423d841061b0f54ef868/users/d5c0292656124047bb3bcf7dbf4eb55a/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:57.129134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-5dab3c2c-2199-4639-acb6-6f96c00462ff None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:57.130290 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10051dc7-60c9-47f0-bae0-ad9ae9b8e3ff None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:57.130645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10051dc7-60c9-47f0-bae0-ad9ae9b8e3ff None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:57.131016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10051dc7-60c9-47f0-bae0-ad9ae9b8e3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:57.131138 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5dab3c2c-2199-4639-acb6-6f96c00462ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:57.152711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:57.236857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-10051dc7-60c9-47f0-bae0-ad9ae9b8e3ff None None] MFA Rules not processed for user `d5c0292656124047bb3bcf7dbf4eb55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:57.242960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-10051dc7-60c9-47f0-bae0-ad9ae9b8e3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:57.279267 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 897/1788] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:57.281916 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab3c2c-2199-4639-acb6-6f96c00462ff None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:57.300994 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5dab3c2c-2199-4639-acb6-6f96c00462ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:57.470111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 892/1789] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:16:57.475624 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10051dc7-60c9-47f0-bae0-ad9ae9b8e3ff None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:57.509414 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10051dc7-60c9-47f0-bae0-ad9ae9b8e3ff None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:57.511480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab3c2c-2199-4639-acb6-6f96c00462ff None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:57.524326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0868ef-7669-4527-9bab-d7c5f65b22a7 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a9c720d5ce6484a8cea554d71f8985d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ae145739a534e75a045d2ffdd69779c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:57.524979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0868ef-7669-4527-9bab-d7c5f65b22a7 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:57.525253 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0868ef-7669-4527-9bab-d7c5f65b22a7 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:57.525525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0868ef-7669-4527-9bab-d7c5f65b22a7 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:57.526654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0868ef-7669-4527-9bab-d7c5f65b22a7 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:57.526654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0868ef-7669-4527-9bab-d7c5f65b22a7 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:57.529881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7400384-4ce0-484f-857c-b60fdb230669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:57.530636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7400384-4ce0-484f-857c-b60fdb230669 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:57.530913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7400384-4ce0-484f-857c-b60fdb230669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:57.531261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7400384-4ce0-484f-857c-b60fdb230669 admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:57.534313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0868ef-7669-4527-9bab-d7c5f65b22a7 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:57.542519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7400384-4ce0-484f-857c-b60fdb230669 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:57.542857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7400384-4ce0-484f-857c-b60fdb230669 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:57.546777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7400384-4ce0-484f-857c-b60fdb230669 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:57.572937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:57.573620 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 898/1790] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:57.574654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0868ef-7669-4527-9bab-d7c5f65b22a7 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:57.584268 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-be0868ef-7669-4527-9bab-d7c5f65b22a7 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:57.592902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 893/1791] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:57.594632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7400384-4ce0-484f-857c-b60fdb230669 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:57.601707 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d7400384-4ce0-484f-857c-b60fdb230669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:57.653115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0868ef-7669-4527-9bab-d7c5f65b22a7 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:57.670725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7400384-4ce0-484f-857c-b60fdb230669 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:57.697828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14348b-9d30-4d69-96d6-dd5add0f8ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:57.699132 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14348b-9d30-4d69-96d6-dd5add0f8ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:57.699837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14348b-9d30-4d69-96d6-dd5add0f8ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:57.700782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14348b-9d30-4d69-96d6-dd5add0f8ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:57.701935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14348b-9d30-4d69-96d6-dd5add0f8ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:57.704292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:57.706532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14348b-9d30-4d69-96d6-dd5add0f8ec1 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:57.710474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:57.721695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:57.721971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] PATH_INFO: `/v3/users/3a38f53091a04544a534184343bf0578` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:57.738729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a38f53091a04544a534184343bf0578)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:57.740773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:57.863294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 899/1792] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:57 2022] POST /identity/v3/projects => generated 383 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:57.865152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14348b-9d30-4d69-96d6-dd5add0f8ec1 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:57.945810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a38f53091a04544a534184343bf0578'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:57.995618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a38f53091a04544a534184343bf0578'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:58.018425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a38f53091a04544a534184343bf0578'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:58.064376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a38f53091a04544a534184343bf0578'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:58.065219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a38f53091a04544a534184343bf0578'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:58.132590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 894/1793] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:16:57 2022] DELETE /identity/v3/users/3a38f53091a04544a534184343bf0578 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:58.133267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:58.151874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:58.228757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14348b-9d30-4d69-96d6-dd5add0f8ec1 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:58.327394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:58.367764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812536d-2fc4-42b0-8bfe-2e5a106cbd03 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:58.375533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698d3b8-8146-4424-a52e-e1069959a0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:58.376333 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698d3b8-8146-4424-a52e-e1069959a0ec admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:58.376719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698d3b8-8146-4424-a52e-e1069959a0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:58.377013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698d3b8-8146-4424-a52e-e1069959a0ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:58.377905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698d3b8-8146-4424-a52e-e1069959a0ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:58.380208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698d3b8-8146-4424-a52e-e1069959a0ec admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:58.392237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8dde2-d935-4fcb-bf67-2976aab1ee43 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a9c720d5ce6484a8cea554d71f8985d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ae145739a534e75a045d2ffdd69779c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:58.393829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8dde2-d935-4fcb-bf67-2976aab1ee43 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:58.394550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8dde2-d935-4fcb-bf67-2976aab1ee43 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:58.395279 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8dde2-d935-4fcb-bf67-2976aab1ee43 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:58.396369 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8dde2-d935-4fcb-bf67-2976aab1ee43 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:58.400819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8dde2-d935-4fcb-bf67-2976aab1ee43 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:58.457525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 895/1794] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:58 2022] POST /identity/v3/projects => generated 401 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:58.462038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 900/1795] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:57 2022] POST /identity/v3/projects => generated 352 bytes in 598 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:58.464194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56134ffd-277c-4465-a989-5653b99b6a67 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:58.467802 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56134ffd-277c-4465-a989-5653b99b6a67 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:58.467930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698d3b8-8146-4424-a52e-e1069959a0ec admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:58.468772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56134ffd-277c-4465-a989-5653b99b6a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:58.478250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1698d3b8-8146-4424-a52e-e1069959a0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:58.510483 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-56134ffd-277c-4465-a989-5653b99b6a67 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:58.513535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-56134ffd-277c-4465-a989-5653b99b6a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:58.622195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 901/1796] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:58.623537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56134ffd-277c-4465-a989-5653b99b6a67 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:58.643304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:58.643898 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56134ffd-277c-4465-a989-5653b99b6a67 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:58.649419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698d3b8-8146-4424-a52e-e1069959a0ec admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:58.656199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9d632-3532-40d8-b2e0-e499f3eaf621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:58.658160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9d632-3532-40d8-b2e0-e499f3eaf621 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:58.659033 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9d632-3532-40d8-b2e0-e499f3eaf621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:58.659433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87eaa9-0802-4a78-9901-1242d163bfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:58.659795 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9d632-3532-40d8-b2e0-e499f3eaf621 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:58.660226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87eaa9-0802-4a78-9901-1242d163bfa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:58.660605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87eaa9-0802-4a78-9901-1242d163bfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:58.660975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87eaa9-0802-4a78-9901-1242d163bfa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:58.661629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9d632-3532-40d8-b2e0-e499f3eaf621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:58.661781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87eaa9-0802-4a78-9901-1242d163bfa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:58.665568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87eaa9-0802-4a78-9901-1242d163bfa4 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:58.670147 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9d632-3532-40d8-b2e0-e499f3eaf621 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:58.704350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 902/1797] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:58 2022] POST /identity/v3/users => generated 301 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:58.705643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87eaa9-0802-4a78-9901-1242d163bfa4 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:58.708816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2f87eaa9-0802-4a78-9901-1242d163bfa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:58.762515 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 896/1798] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:58 2022] POST /identity/v3/users => generated 298 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:58.764234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9d632-3532-40d8-b2e0-e499f3eaf621 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:58.799402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9d632-3532-40d8-b2e0-e499f3eaf621 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:58.818216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d32878-b13b-4bed-aee3-00a21ed36c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:58.819292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d32878-b13b-4bed-aee3-00a21ed36c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:58.820334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d32878-b13b-4bed-aee3-00a21ed36c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:58.820757 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d32878-b13b-4bed-aee3-00a21ed36c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:58.821679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d32878-b13b-4bed-aee3-00a21ed36c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:58.823741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d32878-b13b-4bed-aee3-00a21ed36c61 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:58.878099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 897/1799] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:58.885533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d32878-b13b-4bed-aee3-00a21ed36c61 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:58.908020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d32878-b13b-4bed-aee3-00a21ed36c61 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:58.933198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa225e-b729-4a31-b993-10683c1a8f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:58.934771 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa225e-b729-4a31-b993-10683c1a8f06 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:58.935304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa225e-b729-4a31-b993-10683c1a8f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:58.935743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa225e-b729-4a31-b993-10683c1a8f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:58.936969 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa225e-b729-4a31-b993-10683c1a8f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:58.940352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa225e-b729-4a31-b993-10683c1a8f06 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:58.973377 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 898/1800] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:58.978503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa225e-b729-4a31-b993-10683c1a8f06 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:58.984612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9aaa225e-b729-4a31-b993-10683c1a8f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:58.986682 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87eaa9-0802-4a78-9901-1242d163bfa4 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:59.003097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d17e88-58c9-4a5a-b459-64eb78ab5b3a tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a9c720d5ce6484a8cea554d71f8985d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ae145739a534e75a045d2ffdd69779c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:59.004128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d17e88-58c9-4a5a-b459-64eb78ab5b3a tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:59.005586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d17e88-58c9-4a5a-b459-64eb78ab5b3a tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:59.005995 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d17e88-58c9-4a5a-b459-64eb78ab5b3a tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:59.008725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d17e88-58c9-4a5a-b459-64eb78ab5b3a tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:59.009487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d17e88-58c9-4a5a-b459-64eb78ab5b3a tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:59.012770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d17e88-58c9-4a5a-b459-64eb78ab5b3a tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:59.059763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 903/1801] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:59.066194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d17e88-58c9-4a5a-b459-64eb78ab5b3a tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:59.151464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d17e88-58c9-4a5a-b459-64eb78ab5b3a tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:59.183718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317b13c-8825-403f-a8f3-5611f69db5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:59.188577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317b13c-8825-403f-a8f3-5611f69db5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:59.189219 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317b13c-8825-403f-a8f3-5611f69db5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:59.192574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317b13c-8825-403f-a8f3-5611f69db5d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:59.196151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317b13c-8825-403f-a8f3-5611f69db5d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:59.205018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317b13c-8825-403f-a8f3-5611f69db5d3 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:59.210339 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa225e-b729-4a31-b993-10683c1a8f06 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:59.229588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ef5b0-1eb2-45a0-8084-431d5e3759e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:59.230653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ef5b0-1eb2-45a0-8084-431d5e3759e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:59.231158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ef5b0-1eb2-45a0-8084-431d5e3759e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:59.231676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ef5b0-1eb2-45a0-8084-431d5e3759e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:59.256472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 904/1802] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:59.257663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317b13c-8825-403f-a8f3-5611f69db5d3 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:59.261419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ef5b0-1eb2-45a0-8084-431d5e3759e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:59.263166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ef5b0-1eb2-45a0-8084-431d5e3759e5 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:59.285881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317b13c-8825-403f-a8f3-5611f69db5d3 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:59.290376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 899/1803] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:59.291142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ef5b0-1eb2-45a0-8084-431d5e3759e5 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:59.299181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdb33a-2619-4fb2-9db1-0e1a49573d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:59.299421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdb33a-2619-4fb2-9db1-0e1a49573d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:59.299791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdb33a-2619-4fb2-9db1-0e1a49573d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:59.299791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdb33a-2619-4fb2-9db1-0e1a49573d08 admin admin] PATH_INFO: `/v3/projects/c827d44edbc943c2b9c8b3b8a5228ec9/users/08bcf3d5692345a0b7cb5aeb8214f0f0/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:59.317344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ef5b0-1eb2-45a0-8084-431d5e3759e5 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:59.329073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a9c720d5ce6484a8cea554d71f8985d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ae145739a534e75a045d2ffdd69779c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:59.329661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:59.330005 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:59.330300 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] PATH_INFO: `/v3/projects/0e1318aefe3545509b8593a8c1e731e8` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:59.331518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0e1318aefe3545509b8593a8c1e731e8)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:59.333457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:59.337798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdb33a-2619-4fb2-9db1-0e1a49573d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c827d44edbc943c2b9c8b3b8a5228ec9, user_id=08bcf3d5692345a0b7cb5aeb8214f0f0, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:59.345762 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdb33a-2619-4fb2-9db1-0e1a49573d08 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:59.389598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 905/1804] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:59 2022] PUT /identity/v3/projects/c827d44edbc943c2b9c8b3b8a5228ec9/users/08bcf3d5692345a0b7cb5aeb8214f0f0/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:59.394330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdb33a-2619-4fb2-9db1-0e1a49573d08 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:59.403853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:59.493160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e1318aefe3545509b8593a8c1e731e8', 'request_id': 'req-d6761d02-1122-437a-99b0-6fb2ec7e3f21'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:59.511761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e1318aefe3545509b8593a8c1e731e8', 'request_id': 'req-d6761d02-1122-437a-99b0-6fb2ec7e3f21'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:59.528999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e1318aefe3545509b8593a8c1e731e8', 'request_id': 'req-d6761d02-1122-437a-99b0-6fb2ec7e3f21'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:59.565011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] The token cache is being invalidate because project 0e1318aefe3545509b8593a8c1e731e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:16:59.565536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:59.567976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:59.573729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 900/1805] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:16:59 2022] DELETE /identity/v3/projects/0e1318aefe3545509b8593a8c1e731e8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:59.575109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:59.576839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:59.588131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdb33a-2619-4fb2-9db1-0e1a49573d08 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:59.794381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:59.884392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6761d02-1122-437a-99b0-6fb2ec7e3f21 tempest-VolumeQuotaClassesTest-871401627 tempest-VolumeQuotaClassesTest-871401627-project-admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:59.904344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2f6bb-7d95-4667-9bf0-8381c19e11c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:59.904344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2f6bb-7d95-4667-9bf0-8381c19e11c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:59.904344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2f6bb-7d95-4667-9bf0-8381c19e11c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:59.904344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2f6bb-7d95-4667-9bf0-8381c19e11c1 admin admin] PATH_INFO: `/v3/projects/76ac05546fa44c1f8aff5f3036fc1d47/users/b311c2f122714c909fca2a9fd48fcbc0/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:59.932240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fff5e2-28a3-40f4-b0a7-4088f85a45a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:59.933293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fff5e2-28a3-40f4-b0a7-4088f85a45a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:59.934431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fff5e2-28a3-40f4-b0a7-4088f85a45a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:59.935783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fff5e2-28a3-40f4-b0a7-4088f85a45a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:59.939035 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fff5e2-28a3-40f4-b0a7-4088f85a45a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:59.945541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fff5e2-28a3-40f4-b0a7-4088f85a45a9 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:59.981172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2f6bb-7d95-4667-9bf0-8381c19e11c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ac05546fa44c1f8aff5f3036fc1d47, user_id=b311c2f122714c909fca2a9fd48fcbc0, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:59.998535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2f6bb-7d95-4667-9bf0-8381c19e11c1 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:00.021521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 901/1806] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:00.027453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69caebfc-fe24-4dac-a1c9-3b2ab4319832 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:00.027957 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69caebfc-fe24-4dac-a1c9-3b2ab4319832 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:00.028649 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69caebfc-fe24-4dac-a1c9-3b2ab4319832 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:00.034095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 902/1807] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:17:00 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:17:00.041121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69caebfc-fe24-4dac-a1c9-3b2ab4319832 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:00.088386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 906/1808] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:59 2022] PUT /identity/v3/projects/76ac05546fa44c1f8aff5f3036fc1d47/users/b311c2f122714c909fca2a9fd48fcbc0/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:00.103528 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575fbb3-c42b-4a5f-9cd5-93d160dd7bbe None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:00.104141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575fbb3-c42b-4a5f-9cd5-93d160dd7bbe None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:00.104632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575fbb3-c42b-4a5f-9cd5-93d160dd7bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:00.116087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69caebfc-fe24-4dac-a1c9-3b2ab4319832 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:00.222580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-4575fbb3-c42b-4a5f-9cd5-93d160dd7bbe None None] MFA Rules not processed for user `b311c2f122714c909fca2a9fd48fcbc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:00.223823 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4575fbb3-c42b-4a5f-9cd5-93d160dd7bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:00.353838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6911d-b72c-4e4a-bb9b-5d13a5fb9380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:00.354556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6911d-b72c-4e4a-bb9b-5d13a5fb9380 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:00.354832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6911d-b72c-4e4a-bb9b-5d13a5fb9380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:00.355122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6911d-b72c-4e4a-bb9b-5d13a5fb9380 admin admin] PATH_INFO: `/v3/projects/c827d44edbc943c2b9c8b3b8a5228ec9/users/08bcf3d5692345a0b7cb5aeb8214f0f0/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:00.362427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6911d-b72c-4e4a-bb9b-5d13a5fb9380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c827d44edbc943c2b9c8b3b8a5228ec9, user_id=08bcf3d5692345a0b7cb5aeb8214f0f0, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:00.366857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6911d-b72c-4e4a-bb9b-5d13a5fb9380 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:00.409287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 903/1809] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:00 2022] PUT /identity/v3/projects/c827d44edbc943c2b9c8b3b8a5228ec9/users/08bcf3d5692345a0b7cb5aeb8214f0f0/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:00.412576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f2f4d-dffa-46e6-9d34-43aed42ea3ee None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:00.414301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f2f4d-dffa-46e6-9d34-43aed42ea3ee None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:00.414862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f2f4d-dffa-46e6-9d34-43aed42ea3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:00.468439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-462f2f4d-dffa-46e6-9d34-43aed42ea3ee None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:00.470616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-462f2f4d-dffa-46e6-9d34-43aed42ea3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:00.511093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 907/1810] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:00 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:00.529198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575fbb3-c42b-4a5f-9cd5-93d160dd7bbe None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:00.531056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4575fbb3-c42b-4a5f-9cd5-93d160dd7bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:00.561357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575fbb3-c42b-4a5f-9cd5-93d160dd7bbe None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:00.616144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:00.677159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 904/1811] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:17:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:00.681030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39b888-4885-4b25-9c60-592182e58dee None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:00.681381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39b888-4885-4b25-9c60-592182e58dee None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:00.681689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39b888-4885-4b25-9c60-592182e58dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:00.772094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b77cd-c107-4694-9f29-28497667b2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:00.773358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b77cd-c107-4694-9f29-28497667b2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:00.773654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b77cd-c107-4694-9f29-28497667b2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:00.773932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b77cd-c107-4694-9f29-28497667b2ef admin admin] PATH_INFO: `/v3/users/3544aca3cd8a4a93b6648fb9b24cfe44` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:00.775375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b77cd-c107-4694-9f29-28497667b2ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=3544aca3cd8a4a93b6648fb9b24cfe44)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:00.780313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b77cd-c107-4694-9f29-28497667b2ef admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:00.791816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ab39b888-4885-4b25-9c60-592182e58dee None None] MFA Rules not processed for user `08bcf3d5692345a0b7cb5aeb8214f0f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:00.793305 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ab39b888-4885-4b25-9c60-592182e58dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:00.890817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-9e2b77cd-c107-4694-9f29-28497667b2ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3544aca3cd8a4a93b6648fb9b24cfe44'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:00.903968 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-9e2b77cd-c107-4694-9f29-28497667b2ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3544aca3cd8a4a93b6648fb9b24cfe44'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:00.904690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-9e2b77cd-c107-4694-9f29-28497667b2ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3544aca3cd8a4a93b6648fb9b24cfe44'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:00.906095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-9e2b77cd-c107-4694-9f29-28497667b2ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3544aca3cd8a4a93b6648fb9b24cfe44'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:00.943077 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-9e2b77cd-c107-4694-9f29-28497667b2ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3544aca3cd8a4a93b6648fb9b24cfe44'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:00.956714 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:00.979080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 905/1812] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:00.991390 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 908/1813] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:00 2022] DELETE /identity/v3/users/3544aca3cd8a4a93b6648fb9b24cfe44 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:00.993052 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d8872-3d1e-45ba-862b-260374e90573 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:00.993518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39b888-4885-4b25-9c60-592182e58dee None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:00.994424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d8872-3d1e-45ba-862b-260374e90573 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:00.994738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d8872-3d1e-45ba-862b-260374e90573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:00.998452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ab39b888-4885-4b25-9c60-592182e58dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:01.033669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39b888-4885-4b25-9c60-592182e58dee None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:01.050597 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-638d8872-3d1e-45ba-862b-260374e90573 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:01.051931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-638d8872-3d1e-45ba-862b-260374e90573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:01.172039 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 909/1814] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:01.173026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d8872-3d1e-45ba-862b-260374e90573 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:01.174325 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-638d8872-3d1e-45ba-862b-260374e90573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:01.218152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec283999-faa4-4477-a4a4-8602cb76cf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:01.219062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec283999-faa4-4477-a4a4-8602cb76cf7d admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:01.219485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec283999-faa4-4477-a4a4-8602cb76cf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:01.219893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec283999-faa4-4477-a4a4-8602cb76cf7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:01.220696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec283999-faa4-4477-a4a4-8602cb76cf7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:01.222793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec283999-faa4-4477-a4a4-8602cb76cf7d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:01.272573 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:01.279597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 906/1815] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:00 2022] POST /identity/v3/projects => generated 401 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:01.280778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec283999-faa4-4477-a4a4-8602cb76cf7d admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:01.296982 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ec283999-faa4-4477-a4a4-8602cb76cf7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:01.328095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d8872-3d1e-45ba-862b-260374e90573 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:01.468827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0719ef3-dbaa-4e4a-b08d-4efbbb9c57db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:01.469494 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0719ef3-dbaa-4e4a-b08d-4efbbb9c57db service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:01.469794 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0719ef3-dbaa-4e4a-b08d-4efbbb9c57db service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:01.470082 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0719ef3-dbaa-4e4a-b08d-4efbbb9c57db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:01.478988 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec283999-faa4-4477-a4a4-8602cb76cf7d admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:01.488719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1c442-7219-4254-8ed9-4b1ac7c4ec86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:01.489623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1c442-7219-4254-8ed9-4b1ac7c4ec86 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:01.490074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1c442-7219-4254-8ed9-4b1ac7c4ec86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:01.490493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1c442-7219-4254-8ed9-4b1ac7c4ec86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:01.675251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0719ef3-dbaa-4e4a-b08d-4efbbb9c57db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:01.680398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0719ef3-dbaa-4e4a-b08d-4efbbb9c57db service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:01.698006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1c442-7219-4254-8ed9-4b1ac7c4ec86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:01.701477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1c442-7219-4254-8ed9-4b1ac7c4ec86 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:01.706214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 910/1816] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:01.707243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0719ef3-dbaa-4e4a-b08d-4efbbb9c57db service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:01.726387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 907/1817] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:01.729485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1c442-7219-4254-8ed9-4b1ac7c4ec86 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:01.731868 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b2b1c442-7219-4254-8ed9-4b1ac7c4ec86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:01.732766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0719ef3-dbaa-4e4a-b08d-4efbbb9c57db service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:01.746121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85696a1a-e42e-407c-92d5-f44dbb030edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:01.746121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85696a1a-e42e-407c-92d5-f44dbb030edc admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:01.746121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85696a1a-e42e-407c-92d5-f44dbb030edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:01.746121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85696a1a-e42e-407c-92d5-f44dbb030edc admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:01.746808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85696a1a-e42e-407c-92d5-f44dbb030edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:01.755289 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85696a1a-e42e-407c-92d5-f44dbb030edc admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:01.826701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 911/1818] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:01 2022] POST /identity/v3/users => generated 307 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:01.827777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85696a1a-e42e-407c-92d5-f44dbb030edc admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:01.834038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:01.842804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85696a1a-e42e-407c-92d5-f44dbb030edc admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:01.853244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf693d-d044-4007-ad6f-85f46c19862d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:01.853869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf693d-d044-4007-ad6f-85f46c19862d admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:01.855507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf693d-d044-4007-ad6f-85f46c19862d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:01.855868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf693d-d044-4007-ad6f-85f46c19862d admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:01.856491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf693d-d044-4007-ad6f-85f46c19862d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:01.866852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf693d-d044-4007-ad6f-85f46c19862d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:01.898328 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 912/1819] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:01.900367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf693d-d044-4007-ad6f-85f46c19862d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:01.924369 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf693d-d044-4007-ad6f-85f46c19862d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:01.928632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1c442-7219-4254-8ed9-4b1ac7c4ec86 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:01.936968 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4021f45-e772-466b-8c9b-a1eaedf4ecff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:01.937838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4021f45-e772-466b-8c9b-a1eaedf4ecff service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:01.938385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4021f45-e772-466b-8c9b-a1eaedf4ecff service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:01.938925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4021f45-e772-466b-8c9b-a1eaedf4ecff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:01.940885 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e3e46-2c25-4e82-ab11-76159fbbf603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:01.941793 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e3e46-2c25-4e82-ab11-76159fbbf603 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:01.942096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e3e46-2c25-4e82-ab11-76159fbbf603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:01.943114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e3e46-2c25-4e82-ab11-76159fbbf603 admin admin] PATH_INFO: `/v3/projects/d1208c94753f4960aaaea57f880cd991/users/ec903c7876204887b30ce211ac243b8a/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:01.951802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a4021f45-e772-466b-8c9b-a1eaedf4ecff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:01.969195 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4021f45-e772-466b-8c9b-a1eaedf4ecff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:01.971260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4021f45-e772-466b-8c9b-a1eaedf4ecff service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:02.001945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e3e46-2c25-4e82-ab11-76159fbbf603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1208c94753f4960aaaea57f880cd991, user_id=ec903c7876204887b30ce211ac243b8a, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:02.004211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 908/1820] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:02.005393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4021f45-e772-466b-8c9b-a1eaedf4ecff service nova] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:02.026565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e3e46-2c25-4e82-ab11-76159fbbf603 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:02.034097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4021f45-e772-466b-8c9b-a1eaedf4ecff service nova] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:02.038792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca798b-f355-48b6-abca-59cb302cf19e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:02.040188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca798b-f355-48b6-abca-59cb302cf19e service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:02.040697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca798b-f355-48b6-abca-59cb302cf19e service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:02.041599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca798b-f355-48b6-abca-59cb302cf19e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:02.043834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-30ca798b-f355-48b6-abca-59cb302cf19e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:02.077772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 913/1821] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:01 2022] PUT /identity/v3/projects/d1208c94753f4960aaaea57f880cd991/users/ec903c7876204887b30ce211ac243b8a/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:02.085838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e3e46-2c25-4e82-ab11-76159fbbf603 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:02.126656 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca798b-f355-48b6-abca-59cb302cf19e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:02.128212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca798b-f355-48b6-abca-59cb302cf19e service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:02.142081 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 909/1822] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:02.304799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:02.358390 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e3e46-2c25-4e82-ab11-76159fbbf603 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:02.385737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f7ff4-de05-4673-b3e0-3e676c921b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:02.388975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f7ff4-de05-4673-b3e0-3e676c921b13 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:02.389385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f7ff4-de05-4673-b3e0-3e676c921b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:02.389753 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f7ff4-de05-4673-b3e0-3e676c921b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:02.391180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f7ff4-de05-4673-b3e0-3e676c921b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:02.396544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f7ff4-de05-4673-b3e0-3e676c921b13 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:02.422630 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 914/1823] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:02.426525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca798b-f355-48b6-abca-59cb302cf19e service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:02.444176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca798b-f355-48b6-abca-59cb302cf19e service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:02.458091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da213f-83df-4959-9c8d-0be64976801e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:02.458966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da213f-83df-4959-9c8d-0be64976801e admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:02.459257 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da213f-83df-4959-9c8d-0be64976801e admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:02.459573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da213f-83df-4959-9c8d-0be64976801e admin admin] PATH_INFO: `/v3/projects/d1208c94753f4960aaaea57f880cd991/users/ec903c7876204887b30ce211ac243b8a/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:02.466142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da213f-83df-4959-9c8d-0be64976801e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1208c94753f4960aaaea57f880cd991, user_id=ec903c7876204887b30ce211ac243b8a, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:02.467764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f7ff4-de05-4673-b3e0-3e676c921b13 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:02.471067 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da213f-83df-4959-9c8d-0be64976801e admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:02.472215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a61f7ff4-de05-4673-b3e0-3e676c921b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:02.529484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f7ff4-de05-4673-b3e0-3e676c921b13 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:02.540054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 910/1824] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:02 2022] PUT /identity/v3/projects/d1208c94753f4960aaaea57f880cd991/users/ec903c7876204887b30ce211ac243b8a/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:02.554604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2fa31-72b6-4a7b-992b-0bab60c1d73d None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:02.554998 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2fa31-72b6-4a7b-992b-0bab60c1d73d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:02.555206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2fa31-72b6-4a7b-992b-0bab60c1d73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:02.682049 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-2dd2fa31-72b6-4a7b-992b-0bab60c1d73d None None] MFA Rules not processed for user `ec903c7876204887b30ce211ac243b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:02.683690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2dd2fa31-72b6-4a7b-992b-0bab60c1d73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:02.768385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:02.769711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:02.769711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:02.769884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] PATH_INFO: `/v3/projects/3a9a8831e4f14b72b90d60854a9433ea` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:02.771152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a9a8831e4f14b72b90d60854a9433ea)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:02.774271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:02.891495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 911/1825] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:02 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:02.906223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a9a8831e4f14b72b90d60854a9433ea', 'request_id': 'req-8994f2c5-42ec-434b-8058-6e1e30ebc380'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:02.907106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2fa31-72b6-4a7b-992b-0bab60c1d73d None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:02.909889 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2dd2fa31-72b6-4a7b-992b-0bab60c1d73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:02.915482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a9a8831e4f14b72b90d60854a9433ea', 'request_id': 'req-8994f2c5-42ec-434b-8058-6e1e30ebc380'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:02.927561 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a9a8831e4f14b72b90d60854a9433ea', 'request_id': 'req-8994f2c5-42ec-434b-8058-6e1e30ebc380'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:02.951106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] The token cache is being invalidate because project 3a9a8831e4f14b72b90d60854a9433ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:02.951311 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:02.953545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:02.956765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 915/1826] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:02 2022] DELETE /identity/v3/projects/3a9a8831e4f14b72b90d60854a9433ea => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:02.961306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:02.962787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:02.971137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:03.027620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2fa31-72b6-4a7b-992b-0bab60c1d73d None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:03.130691 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994f2c5-42ec-434b-8058-6e1e30ebc380 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:03.137585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:03.138254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.138532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.138809 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] PATH_INFO: `/v3/users/bb532d212e11401098c306e088719a52` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.141389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb532d212e11401098c306e088719a52)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:03.146185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:03.161550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9d295-1d20-4d95-97e6-8e6f5ec9ccc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:03.162321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9d295-1d20-4d95-97e6-8e6f5ec9ccc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.164618 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9d295-1d20-4d95-97e6-8e6f5ec9ccc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.166790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9d295-1d20-4d95-97e6-8e6f5ec9ccc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.177467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-20b9d295-1d20-4d95-97e6-8e6f5ec9ccc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:03.218633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:03.266516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb532d212e11401098c306e088719a52'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:03.280785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb532d212e11401098c306e088719a52'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:03.281714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb532d212e11401098c306e088719a52'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:03.283009 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb532d212e11401098c306e088719a52'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:03.329952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb532d212e11401098c306e088719a52'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:03.354667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 916/1827] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:02 2022] DELETE /identity/v3/users/bb532d212e11401098c306e088719a52 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:03.373736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9d295-1d20-4d95-97e6-8e6f5ec9ccc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:03.375594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9d295-1d20-4d95-97e6-8e6f5ec9ccc3 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:03.387826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 912/1828] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:02 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:03.464581 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:03.466387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:03.553313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9d295-1d20-4d95-97e6-8e6f5ec9ccc3 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:03.556029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-20b9d295-1d20-4d95-97e6-8e6f5ec9ccc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:03.578885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9d295-1d20-4d95-97e6-8e6f5ec9ccc3 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:03.629296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb16f5-0e0f-4cb9-b24e-49dad465c596 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:03.645247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:03.645247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.645247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.645247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] PATH_INFO: `/v3/projects/89ff3113974a442c899ae9a54df289eb` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.646449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ff3113974a442c899ae9a54df289eb)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:03.648913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:03.683186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41543b58-0506-484e-b68a-0a7b70547d5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:03.683599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41543b58-0506-484e-b68a-0a7b70547d5d service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.683879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41543b58-0506-484e-b68a-0a7b70547d5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.684185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41543b58-0506-484e-b68a-0a7b70547d5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.690105 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-41543b58-0506-484e-b68a-0a7b70547d5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:03.730198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ff3113974a442c899ae9a54df289eb', 'request_id': 'req-aa296f99-9969-4337-89dd-566de4fa96e2'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:03.740972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ff3113974a442c899ae9a54df289eb', 'request_id': 'req-aa296f99-9969-4337-89dd-566de4fa96e2'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:03.759139 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ff3113974a442c899ae9a54df289eb', 'request_id': 'req-aa296f99-9969-4337-89dd-566de4fa96e2'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:03.770511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:03.786371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] The token cache is being invalidate because project 89ff3113974a442c899ae9a54df289eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:03.786968 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:03.788633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:03.794287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 917/1829] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:03 2022] DELETE /identity/v3/projects/89ff3113974a442c899ae9a54df289eb => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:03.795545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:03.798049 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:03.799941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41543b58-0506-484e-b68a-0a7b70547d5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:03.801513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41543b58-0506-484e-b68a-0a7b70547d5d service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:03.828094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 913/1830] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:03 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:03.870128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:03.948168 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa296f99-9969-4337-89dd-566de4fa96e2 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:03.972852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64290541-2d56-4e26-a22a-da4c439c15df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:03.973512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64290541-2d56-4e26-a22a-da4c439c15df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.973797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64290541-2d56-4e26-a22a-da4c439c15df admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.974070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64290541-2d56-4e26-a22a-da4c439c15df admin admin] PATH_INFO: `/v3/users/0a9c720d5ce6484a8cea554d71f8985d` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.975727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64290541-2d56-4e26-a22a-da4c439c15df admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a9c720d5ce6484a8cea554d71f8985d)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:03.977523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64290541-2d56-4e26-a22a-da4c439c15df admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:04.099720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-64290541-2d56-4e26-a22a-da4c439c15df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a9c720d5ce6484a8cea554d71f8985d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:04.117719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-64290541-2d56-4e26-a22a-da4c439c15df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a9c720d5ce6484a8cea554d71f8985d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:04.120555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-64290541-2d56-4e26-a22a-da4c439c15df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a9c720d5ce6484a8cea554d71f8985d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:04.121391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-64290541-2d56-4e26-a22a-da4c439c15df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a9c720d5ce6484a8cea554d71f8985d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:04.166880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-64290541-2d56-4e26-a22a-da4c439c15df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a9c720d5ce6484a8cea554d71f8985d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:04.189707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 918/1831] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:03 2022] DELETE /identity/v3/users/0a9c720d5ce6484a8cea554d71f8985d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:04.393939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41543b58-0506-484e-b68a-0a7b70547d5d service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:04.399349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-41543b58-0506-484e-b68a-0a7b70547d5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:04.435535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41543b58-0506-484e-b68a-0a7b70547d5d service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:04.596051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:04.600234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:04.602042 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:04.602688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] PATH_INFO: `/v3/projects/ee48e31c633544009aeb6124fbb0bd4a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:04.605309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee48e31c633544009aeb6124fbb0bd4a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:04.608475 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:04.711840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee48e31c633544009aeb6124fbb0bd4a', 'request_id': 'req-d0d69301-d1a7-438e-b6b5-a131f4f56550'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:04.723897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee48e31c633544009aeb6124fbb0bd4a', 'request_id': 'req-d0d69301-d1a7-438e-b6b5-a131f4f56550'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:04.734077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee48e31c633544009aeb6124fbb0bd4a', 'request_id': 'req-d0d69301-d1a7-438e-b6b5-a131f4f56550'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:04.751604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] The token cache is being invalidate because project ee48e31c633544009aeb6124fbb0bd4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:04.751981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:04.757466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:04.759466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 914/1832] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:04 2022] DELETE /identity/v3/projects/ee48e31c633544009aeb6124fbb0bd4a => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:04.768781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec847f00-0f95-44c4-8c94-a755ccd62be3 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:04.769437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec847f00-0f95-44c4-8c94-a755ccd62be3 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:04.770076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec847f00-0f95-44c4-8c94-a755ccd62be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:04.792816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:04.794439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:04.808164 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-ec847f00-0f95-44c4-8c94-a755ccd62be3 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:04.810198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ec847f00-0f95-44c4-8c94-a755ccd62be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:04.925379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d69301-d1a7-438e-b6b5-a131f4f56550 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:04.931470 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 919/1833] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:04.934454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f19e2-a339-4645-980f-97af07a8f4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:04.935677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f19e2-a339-4645-980f-97af07a8f4d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:04.936544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f19e2-a339-4645-980f-97af07a8f4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:04.937120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f19e2-a339-4645-980f-97af07a8f4d7 admin admin] PATH_INFO: `/v3/projects/8ae145739a534e75a045d2ffdd69779c` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:04.940380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43727426-e2d8-49eb-9401-1fb206fb8495 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:04.940591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43727426-e2d8-49eb-9401-1fb206fb8495 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:04.941027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43727426-e2d8-49eb-9401-1fb206fb8495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:04.941423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f19e2-a339-4645-980f-97af07a8f4d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae145739a534e75a045d2ffdd69779c)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:04.943543 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f19e2-a339-4645-980f-97af07a8f4d7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:04.965382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-43727426-e2d8-49eb-9401-1fb206fb8495 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:04.966663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-43727426-e2d8-49eb-9401-1fb206fb8495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:04.972092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:04.978563 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 920/1834] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:04.982751 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43727426-e2d8-49eb-9401-1fb206fb8495 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:04.999307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43727426-e2d8-49eb-9401-1fb206fb8495 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.005850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b94f19e2-a339-4645-980f-97af07a8f4d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae145739a534e75a045d2ffdd69779c', 'request_id': 'req-b94f19e2-a339-4645-980f-97af07a8f4d7'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:05.019728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b94f19e2-a339-4645-980f-97af07a8f4d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae145739a534e75a045d2ffdd69779c', 'request_id': 'req-b94f19e2-a339-4645-980f-97af07a8f4d7'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:05.028947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b94f19e2-a339-4645-980f-97af07a8f4d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae145739a534e75a045d2ffdd69779c', 'request_id': 'req-b94f19e2-a339-4645-980f-97af07a8f4d7'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:05.045611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b94f19e2-a339-4645-980f-97af07a8f4d7 admin admin] The token cache is being invalidate because project 8ae145739a534e75a045d2ffdd69779c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:05.046043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b94f19e2-a339-4645-980f-97af07a8f4d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:05.047218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b94f19e2-a339-4645-980f-97af07a8f4d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:05.052169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 915/1835] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:04 2022] DELETE /identity/v3/projects/8ae145739a534e75a045d2ffdd69779c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:05.056078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee689-b388-4f66-9fd0-45e81a1f7b46 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.057522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee689-b388-4f66-9fd0-45e81a1f7b46 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.057522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee689-b388-4f66-9fd0-45e81a1f7b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.067098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:05.090626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ad5ee689-b388-4f66-9fd0-45e81a1f7b46 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:05.092020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ad5ee689-b388-4f66-9fd0-45e81a1f7b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:05.092443 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa4450-1bbd-4b80-90d4-7ac0fc953cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.093076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa4450-1bbd-4b80-90d4-7ac0fc953cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.093361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa4450-1bbd-4b80-90d4-7ac0fc953cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.093636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa4450-1bbd-4b80-90d4-7ac0fc953cf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.094296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa4450-1bbd-4b80-90d4-7ac0fc953cf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:05.094608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa4450-1bbd-4b80-90d4-7ac0fc953cf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.096514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa4450-1bbd-4b80-90d4-7ac0fc953cf5 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.103812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 916/1836] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:05.112094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3115b49-6cd0-454b-a879-db8ab14ca35e None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.112439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3115b49-6cd0-454b-a879-db8ab14ca35e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.112727 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3115b49-6cd0-454b-a879-db8ab14ca35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.141710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 921/1837] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:05.146376 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa4450-1bbd-4b80-90d4-7ac0fc953cf5 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.148968 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b6aa4450-1bbd-4b80-90d4-7ac0fc953cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:05.155942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e3115b49-6cd0-454b-a879-db8ab14ca35e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:05.157343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e3115b49-6cd0-454b-a879-db8ab14ca35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:05.162253 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa4450-1bbd-4b80-90d4-7ac0fc953cf5 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.167931 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 917/1838] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:05.170400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0aaf5-90f5-4d9f-b12a-93b798e2d313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.170400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0aaf5-90f5-4d9f-b12a-93b798e2d313 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.170794 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0aaf5-90f5-4d9f-b12a-93b798e2d313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.171103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0aaf5-90f5-4d9f-b12a-93b798e2d313 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.171765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0aaf5-90f5-4d9f-b12a-93b798e2d313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.172549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3115b49-6cd0-454b-a879-db8ab14ca35e None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.173846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0aaf5-90f5-4d9f-b12a-93b798e2d313 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.187478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3115b49-6cd0-454b-a879-db8ab14ca35e None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.193580 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fc20a-efb6-454b-a30c-5ae21e6dd8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.194986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fc20a-efb6-454b-a30c-5ae21e6dd8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.195477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fc20a-efb6-454b-a30c-5ae21e6dd8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.196011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fc20a-efb6-454b-a30c-5ae21e6dd8f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.196839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fc20a-efb6-454b-a30c-5ae21e6dd8f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:05.197237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fc20a-efb6-454b-a30c-5ae21e6dd8f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.198929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fc20a-efb6-454b-a30c-5ae21e6dd8f4 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.215340 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 922/1839] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:05 2022] POST /identity/v3/projects => generated 369 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:05.216168 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0aaf5-90f5-4d9f-b12a-93b798e2d313 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.217782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 918/1840] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:05.222121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fc20a-efb6-454b-a30c-5ae21e6dd8f4 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.248880 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fc20a-efb6-454b-a30c-5ae21e6dd8f4 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.297741 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0aaf5-90f5-4d9f-b12a-93b798e2d313 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.299932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2940ad-653b-460d-a275-6c3ebafe7829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.300683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2940ad-653b-460d-a275-6c3ebafe7829 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.300963 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2940ad-653b-460d-a275-6c3ebafe7829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.301233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2940ad-653b-460d-a275-6c3ebafe7829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.301880 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2940ad-653b-460d-a275-6c3ebafe7829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.302326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09620f9a-1187-400b-b6b9-041b54701e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.302880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09620f9a-1187-400b-b6b9-041b54701e60 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.303197 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09620f9a-1187-400b-b6b9-041b54701e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.303621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09620f9a-1187-400b-b6b9-041b54701e60 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.305107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2940ad-653b-460d-a275-6c3ebafe7829 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.305247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09620f9a-1187-400b-b6b9-041b54701e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.306800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09620f9a-1187-400b-b6b9-041b54701e60 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.330299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 923/1841] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:05 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:05.335134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09620f9a-1187-400b-b6b9-041b54701e60 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.341351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 919/1842] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:05 2022] POST /identity/v3/projects => generated 391 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:05.342428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2940ad-653b-460d-a275-6c3ebafe7829 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.346683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09620f9a-1187-400b-b6b9-041b54701e60 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.416719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2940ad-653b-460d-a275-6c3ebafe7829 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.422658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe22ed8-902b-4c45-8fbf-5572a41fc74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.423320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe22ed8-902b-4c45-8fbf-5572a41fc74f admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.423588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe22ed8-902b-4c45-8fbf-5572a41fc74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.423974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe22ed8-902b-4c45-8fbf-5572a41fc74f admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.424734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe22ed8-902b-4c45-8fbf-5572a41fc74f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.428088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe22ed8-902b-4c45-8fbf-5572a41fc74f admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.431730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94941b-405b-4bf1-87df-fbf7e643119d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.432731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94941b-405b-4bf1-87df-fbf7e643119d admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.433162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94941b-405b-4bf1-87df-fbf7e643119d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.433664 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94941b-405b-4bf1-87df-fbf7e643119d admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.434481 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da94941b-405b-4bf1-87df-fbf7e643119d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.436529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da94941b-405b-4bf1-87df-fbf7e643119d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.453511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 924/1843] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:05.458421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94941b-405b-4bf1-87df-fbf7e643119d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.462121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:05.469903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 920/1844] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:05 2022] POST /identity/v3/users => generated 296 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:05.474998 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe22ed8-902b-4c45-8fbf-5572a41fc74f admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.478446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94941b-405b-4bf1-87df-fbf7e643119d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.489838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe22ed8-902b-4c45-8fbf-5572a41fc74f admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.498707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f525533-a969-4191-bc97-82cfa666e375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.499587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f525533-a969-4191-bc97-82cfa666e375 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.500131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f525533-a969-4191-bc97-82cfa666e375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.500509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f525533-a969-4191-bc97-82cfa666e375 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.501105 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7275a-464b-414a-a7aa-2857980acbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.501445 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f525533-a969-4191-bc97-82cfa666e375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.502088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7275a-464b-414a-a7aa-2857980acbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.502565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7275a-464b-414a-a7aa-2857980acbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.502981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7275a-464b-414a-a7aa-2857980acbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.503737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f525533-a969-4191-bc97-82cfa666e375 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.504091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7275a-464b-414a-a7aa-2857980acbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.506163 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7275a-464b-414a-a7aa-2857980acbd0 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.522678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 921/1845] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:05.526546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 925/1846] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:05.526863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7275a-464b-414a-a7aa-2857980acbd0 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.527468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f525533-a969-4191-bc97-82cfa666e375 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.539451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7275a-464b-414a-a7aa-2857980acbd0 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.545486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f138ff-f8b1-42ec-9226-433534bd3e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.546113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f138ff-f8b1-42ec-9226-433534bd3e4e admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.546397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f138ff-f8b1-42ec-9226-433534bd3e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.546830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f138ff-f8b1-42ec-9226-433534bd3e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.547485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f138ff-f8b1-42ec-9226-433534bd3e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.548456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f525533-a969-4191-bc97-82cfa666e375 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.549288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f138ff-f8b1-42ec-9226-433534bd3e4e admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.555562 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2025b-8356-4286-8686-562032029fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.556959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2025b-8356-4286-8686-562032029fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.557347 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2025b-8356-4286-8686-562032029fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.557639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2025b-8356-4286-8686-562032029fa0 admin admin] PATH_INFO: `/v3/projects/760804bf13744adeb30438d7c75a5a8e/users/8840c29d52bf4aba946243e205b75e23/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.562901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 922/1847] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:05.567010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f138ff-f8b1-42ec-9226-433534bd3e4e admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.578417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f138ff-f8b1-42ec-9226-433534bd3e4e admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.580911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2025b-8356-4286-8686-562032029fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=760804bf13744adeb30438d7c75a5a8e, user_id=8840c29d52bf4aba946243e205b75e23, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.583593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2025b-8356-4286-8686-562032029fa0 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.591198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c729a-d25b-441f-a020-ade09744b9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.591888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c729a-d25b-441f-a020-ade09744b9d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.592198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c729a-d25b-441f-a020-ade09744b9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.592473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c729a-d25b-441f-a020-ade09744b9d8 admin admin] PATH_INFO: `/v3/projects/bae8208c34cf46ed9495a5fdbb522209/users/1c040a27bd92494db1fad3b7ae7fba45/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.611119 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 926/1848] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:05 2022] PUT /identity/v3/projects/760804bf13744adeb30438d7c75a5a8e/users/8840c29d52bf4aba946243e205b75e23/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:05.614589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c729a-d25b-441f-a020-ade09744b9d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bae8208c34cf46ed9495a5fdbb522209, user_id=1c040a27bd92494db1fad3b7ae7fba45, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.617905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c729a-d25b-441f-a020-ade09744b9d8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.622567 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f93f3-7c68-475c-b36b-67639593d200 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.622938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f93f3-7c68-475c-b36b-67639593d200 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.623241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f93f3-7c68-475c-b36b-67639593d200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.636878 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 923/1849] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:05 2022] PUT /identity/v3/projects/bae8208c34cf46ed9495a5fdbb522209/users/1c040a27bd92494db1fad3b7ae7fba45/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:05.648153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:05.648369 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b5d37-04fd-443d-a0f9-dc37d7452369 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.648621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b5d37-04fd-443d-a0f9-dc37d7452369 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.648893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b5d37-04fd-443d-a0f9-dc37d7452369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.674375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-a85f93f3-7c68-475c-b36b-67639593d200 None None] MFA Rules not processed for user `8840c29d52bf4aba946243e205b75e23`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:05.675926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a85f93f3-7c68-475c-b36b-67639593d200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:05.693241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ef4b5d37-04fd-443d-a0f9-dc37d7452369 None None] MFA Rules not processed for user `1c040a27bd92494db1fad3b7ae7fba45`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:05.694536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ef4b5d37-04fd-443d-a0f9-dc37d7452369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:05.788032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 927/1850] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:05.793979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f93f3-7c68-475c-b36b-67639593d200 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.814496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 924/1851] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:05.829970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b5d37-04fd-443d-a0f9-dc37d7452369 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.895771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f93f3-7c68-475c-b36b-67639593d200 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.903356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211b43d-2516-4c20-a563-4943ceb2bc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.904231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211b43d-2516-4c20-a563-4943ceb2bc6f admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.904581 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211b43d-2516-4c20-a563-4943ceb2bc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.905133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211b43d-2516-4c20-a563-4943ceb2bc6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.905866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211b43d-2516-4c20-a563-4943ceb2bc6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.909522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211b43d-2516-4c20-a563-4943ceb2bc6f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:05.989837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 928/1852] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:05 2022] POST /identity/v3/projects => generated 369 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:05.993533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211b43d-2516-4c20-a563-4943ceb2bc6f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:06.009109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1211b43d-2516-4c20-a563-4943ceb2bc6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:06.037518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b5d37-04fd-443d-a0f9-dc37d7452369 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:06.128477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:06.176997 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a2b5a-1d9b-419f-a7ba-c78d4dbd04fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:06.177380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a2b5a-1d9b-419f-a7ba-c78d4dbd04fa admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.178193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a2b5a-1d9b-419f-a7ba-c78d4dbd04fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.178193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a2b5a-1d9b-419f-a7ba-c78d4dbd04fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.178567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a2b5a-1d9b-419f-a7ba-c78d4dbd04fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:06.184220 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a2b5a-1d9b-419f-a7ba-c78d4dbd04fa admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:06.213702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211b43d-2516-4c20-a563-4943ceb2bc6f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:06.218526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:06.219145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.219465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.219701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] PATH_INFO: `/v3/users/d5c0292656124047bb3bcf7dbf4eb55a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.221052 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c0292656124047bb3bcf7dbf4eb55a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:06.222847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:06.231778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 925/1853] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:05 2022] POST /identity/v3/projects => generated 393 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:06.235833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a2b5a-1d9b-419f-a7ba-c78d4dbd04fa admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:06.253776 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:06.297424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c0292656124047bb3bcf7dbf4eb55a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:06.308398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c0292656124047bb3bcf7dbf4eb55a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:06.309732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c0292656124047bb3bcf7dbf4eb55a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:06.310835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c0292656124047bb3bcf7dbf4eb55a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:06.338195 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c0292656124047bb3bcf7dbf4eb55a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:06.353224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a2b5a-1d9b-419f-a7ba-c78d4dbd04fa admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:06.355217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 929/1854] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:05 2022] DELETE /identity/v3/users/d5c0292656124047bb3bcf7dbf4eb55a => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:06.355450 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:06.358615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:06.440933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0ad43-ead0-46bc-9dc4-3c32ed13808b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:06.442048 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0ad43-ead0-46bc-9dc4-3c32ed13808b admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.442497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0ad43-ead0-46bc-9dc4-3c32ed13808b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.442904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0ad43-ead0-46bc-9dc4-3c32ed13808b admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.443555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0ad43-ead0-46bc-9dc4-3c32ed13808b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:06.446396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0ad43-ead0-46bc-9dc4-3c32ed13808b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:06.490051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 926/1855] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:06 2022] POST /identity/v3/users => generated 291 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:06.492737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407845c-2780-415d-aaab-546d51c10a87 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.493026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407845c-2780-415d-aaab-546d51c10a87 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.493300 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407845c-2780-415d-aaab-546d51c10a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.521416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9186a-ae1a-4a54-b32e-b68b4388086b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:06.525691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-0407845c-2780-415d-aaab-546d51c10a87 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:06.528553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0407845c-2780-415d-aaab-546d51c10a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:06.537265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73db6a-cc4a-4fcf-bc35-3a3230088564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:06.540901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73db6a-cc4a-4fcf-bc35-3a3230088564 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.541318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73db6a-cc4a-4fcf-bc35-3a3230088564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.541742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73db6a-cc4a-4fcf-bc35-3a3230088564 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.542678 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73db6a-cc4a-4fcf-bc35-3a3230088564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:06.545068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73db6a-cc4a-4fcf-bc35-3a3230088564 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:06.556157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 927/1856] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:06.556157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407845c-2780-415d-aaab-546d51c10a87 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:06.558271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0407845c-2780-415d-aaab-546d51c10a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:06.574825 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 930/1857] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:06 2022] POST /identity/v3/users => generated 303 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:06.576050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73db6a-cc4a-4fcf-bc35-3a3230088564 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:06.577737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-bb73db6a-cc4a-4fcf-bc35-3a3230088564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:06.586995 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407845c-2780-415d-aaab-546d51c10a87 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:06.595156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2f42a-810c-42e4-b1b6-c6de45d25803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:06.596705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2f42a-810c-42e4-b1b6-c6de45d25803 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.597196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2f42a-810c-42e4-b1b6-c6de45d25803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.597538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2f42a-810c-42e4-b1b6-c6de45d25803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.598853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2f42a-810c-42e4-b1b6-c6de45d25803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:06.604122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2f42a-810c-42e4-b1b6-c6de45d25803 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:06.626948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 928/1858] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:06.628762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2f42a-810c-42e4-b1b6-c6de45d25803 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:06.650906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2f42a-810c-42e4-b1b6-c6de45d25803 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:06.660599 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73db6a-cc4a-4fcf-bc35-3a3230088564 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:06.661900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc11639-9e33-4617-ac14-00537e972ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:06.663536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc11639-9e33-4617-ac14-00537e972ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.664051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc11639-9e33-4617-ac14-00537e972ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.664571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc11639-9e33-4617-ac14-00537e972ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.665673 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765f7ee-3705-4f18-b61c-b11d68be1f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:06.665801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc11639-9e33-4617-ac14-00537e972ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:06.666399 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765f7ee-3705-4f18-b61c-b11d68be1f80 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.666750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765f7ee-3705-4f18-b61c-b11d68be1f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.667098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765f7ee-3705-4f18-b61c-b11d68be1f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.669103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc11639-9e33-4617-ac14-00537e972ad1 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:06.678446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765f7ee-3705-4f18-b61c-b11d68be1f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:06.680275 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765f7ee-3705-4f18-b61c-b11d68be1f80 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:06.682761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 929/1859] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:06.687616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc11639-9e33-4617-ac14-00537e972ad1 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:06.699622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc11639-9e33-4617-ac14-00537e972ad1 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:06.702632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 931/1860] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:06.704085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765f7ee-3705-4f18-b61c-b11d68be1f80 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:06.714894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595faf7-935d-4d2a-ac8e-a569dfbd7bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:06.717858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595faf7-935d-4d2a-ac8e-a569dfbd7bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.719248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595faf7-935d-4d2a-ac8e-a569dfbd7bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.719805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595faf7-935d-4d2a-ac8e-a569dfbd7bcd admin admin] PATH_INFO: `/v3/projects/e6330b3eeabf45cb83066cd401e24460/users/55df8ee8d69d482395c7582299d75285/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.723814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765f7ee-3705-4f18-b61c-b11d68be1f80 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:06.735242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdf395-7384-47cb-9c63-6defd05e0648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:06.737141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdf395-7384-47cb-9c63-6defd05e0648 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.738003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdf395-7384-47cb-9c63-6defd05e0648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.738984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdf395-7384-47cb-9c63-6defd05e0648 admin admin] PATH_INFO: `/v3/projects/cd8b69f0275843da9386e6be64bce685/users/6deba2e3c3a1423ea863f0e46ce79f86/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.774115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595faf7-935d-4d2a-ac8e-a569dfbd7bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6330b3eeabf45cb83066cd401e24460, user_id=55df8ee8d69d482395c7582299d75285, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:06.776749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdf395-7384-47cb-9c63-6defd05e0648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8b69f0275843da9386e6be64bce685, user_id=6deba2e3c3a1423ea863f0e46ce79f86, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:06.780965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595faf7-935d-4d2a-ac8e-a569dfbd7bcd admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:06.787981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdf395-7384-47cb-9c63-6defd05e0648 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:06.818671 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 930/1861] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:06 2022] PUT /identity/v3/projects/e6330b3eeabf45cb83066cd401e24460/users/55df8ee8d69d482395c7582299d75285/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:06.821757 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595faf7-935d-4d2a-ac8e-a569dfbd7bcd admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:06.825727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 932/1862] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:06 2022] PUT /identity/v3/projects/cd8b69f0275843da9386e6be64bce685/users/6deba2e3c3a1423ea863f0e46ce79f86/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:06.834402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdf395-7384-47cb-9c63-6defd05e0648 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:06.863728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:06.883272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:07.002798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdf395-7384-47cb-9c63-6defd05e0648 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:07.008342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595faf7-935d-4d2a-ac8e-a569dfbd7bcd admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:07.019025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91173689-d64d-4b80-8d7a-d403057ae5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:07.020699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91173689-d64d-4b80-8d7a-d403057ae5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:07.021501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91173689-d64d-4b80-8d7a-d403057ae5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:07.022924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91173689-d64d-4b80-8d7a-d403057ae5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:07.024395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91173689-d64d-4b80-8d7a-d403057ae5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:07.028483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91173689-d64d-4b80-8d7a-d403057ae5a7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:07.032577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c5745-7f1c-49d3-bdd5-6e62f2e13877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:07.032577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c5745-7f1c-49d3-bdd5-6e62f2e13877 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:07.032577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c5745-7f1c-49d3-bdd5-6e62f2e13877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:07.032577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c5745-7f1c-49d3-bdd5-6e62f2e13877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:07.033211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c5745-7f1c-49d3-bdd5-6e62f2e13877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:07.033336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c5745-7f1c-49d3-bdd5-6e62f2e13877 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:07.048051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 933/1863] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:07.051160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 931/1864] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:07.056431 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c5745-7f1c-49d3-bdd5-6e62f2e13877 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:07.057637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91173689-d64d-4b80-8d7a-d403057ae5a7 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:07.069719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c5745-7f1c-49d3-bdd5-6e62f2e13877 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:07.076879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa885e9-ad3e-4398-89ae-2b973f24a5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:07.077876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa885e9-ad3e-4398-89ae-2b973f24a5a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:07.078161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa885e9-ad3e-4398-89ae-2b973f24a5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:07.078439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa885e9-ad3e-4398-89ae-2b973f24a5a6 admin admin] PATH_INFO: `/v3/projects/cd8b69f0275843da9386e6be64bce685/users/6deba2e3c3a1423ea863f0e46ce79f86/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:07.080666 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa885e9-ad3e-4398-89ae-2b973f24a5a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8b69f0275843da9386e6be64bce685, user_id=6deba2e3c3a1423ea863f0e46ce79f86, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:07.082851 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91173689-d64d-4b80-8d7a-d403057ae5a7 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:07.083820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa885e9-ad3e-4398-89ae-2b973f24a5a6 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:07.093949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fffb1-3ad7-4c0a-93a7-27f32fe65bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:07.095952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fffb1-3ad7-4c0a-93a7-27f32fe65bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:07.096982 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fffb1-3ad7-4c0a-93a7-27f32fe65bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:07.097507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fffb1-3ad7-4c0a-93a7-27f32fe65bcf admin admin] PATH_INFO: `/v3/projects/e6330b3eeabf45cb83066cd401e24460/users/55df8ee8d69d482395c7582299d75285/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:07.100914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 934/1865] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:07 2022] PUT /identity/v3/projects/cd8b69f0275843da9386e6be64bce685/users/6deba2e3c3a1423ea863f0e46ce79f86/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:07.101399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91fffb1-3ad7-4c0a-93a7-27f32fe65bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6330b3eeabf45cb83066cd401e24460, user_id=55df8ee8d69d482395c7582299d75285, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:07.120423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91fffb1-3ad7-4c0a-93a7-27f32fe65bcf admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:07.120584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314972e-b4e6-44db-933c-fba11335ee77 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:07.120584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314972e-b4e6-44db-933c-fba11335ee77 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:07.120584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314972e-b4e6-44db-933c-fba11335ee77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:07.149297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 932/1866] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:07 2022] PUT /identity/v3/projects/e6330b3eeabf45cb83066cd401e24460/users/55df8ee8d69d482395c7582299d75285/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:07.153882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6774b-6139-443b-ba07-966c79283354 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:07.153882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6774b-6139-443b-ba07-966c79283354 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:07.153882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6774b-6139-443b-ba07-966c79283354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:07.183761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-4314972e-b4e6-44db-933c-fba11335ee77 None None] MFA Rules not processed for user `6deba2e3c3a1423ea863f0e46ce79f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:07.188698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4314972e-b4e6-44db-933c-fba11335ee77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:07.216672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-78a6774b-6139-443b-ba07-966c79283354 None None] MFA Rules not processed for user `55df8ee8d69d482395c7582299d75285`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:07.217932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-78a6774b-6139-443b-ba07-966c79283354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:07.364063 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 935/1867] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:07 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:07.365073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314972e-b4e6-44db-933c-fba11335ee77 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:07.370262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4314972e-b4e6-44db-933c-fba11335ee77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:07.414628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 933/1868] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:07.414962 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6774b-6139-443b-ba07-966c79283354 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:07.416494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-78a6774b-6139-443b-ba07-966c79283354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:07.504146 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314972e-b4e6-44db-933c-fba11335ee77 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:07.515728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:07.517104 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:07.517357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:07.517629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] PATH_INFO: `/v3/projects/d5e393097580423d841061b0f54ef868` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:07.519859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e393097580423d841061b0f54ef868)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:07.521921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:07.579604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:07.582037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6774b-6139-443b-ba07-966c79283354 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:07.597161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70fad3-6389-41b3-a462-e8d284eb62ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:07.598311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70fad3-6389-41b3-a462-e8d284eb62ba service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:07.599104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70fad3-6389-41b3-a462-e8d284eb62ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:07.599706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70fad3-6389-41b3-a462-e8d284eb62ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:07.614907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70fad3-6389-41b3-a462-e8d284eb62ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:07.617970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70fad3-6389-41b3-a462-e8d284eb62ba service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:07.636173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 934/1869] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:07 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:07.640859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e393097580423d841061b0f54ef868', 'request_id': 'req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.649208 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e393097580423d841061b0f54ef868', 'request_id': 'req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.658688 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e393097580423d841061b0f54ef868', 'request_id': 'req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.685645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] The token cache is being invalidate because project d5e393097580423d841061b0f54ef868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:07.686487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.692272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.698692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 936/1870] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:07 2022] DELETE /identity/v3/projects/d5e393097580423d841061b0f54ef868 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:07.699745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70fad3-6389-41b3-a462-e8d284eb62ba service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:07.721083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70fad3-6389-41b3-a462-e8d284eb62ba service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:07.748882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:07.752748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:07.815474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:07.929206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:07.931037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:07.931529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:07.932023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] PATH_INFO: `/v3/users/08bcf3d5692345a0b7cb5aeb8214f0f0` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:07.934272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08bcf3d5692345a0b7cb5aeb8214f0f0)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:07.937348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:08.001367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866cd3a-53b6-4b8d-aecc-3559a013b4d8 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:08.014318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba63ee-eaf7-4d68-bff5-0bf5488a47ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:08.015304 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba63ee-eaf7-4d68-bff5-0bf5488a47ff service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:08.015665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba63ee-eaf7-4d68-bff5-0bf5488a47ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:08.017517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba63ee-eaf7-4d68-bff5-0bf5488a47ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:08.021138 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-71ba63ee-eaf7-4d68-bff5-0bf5488a47ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:08.024548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08bcf3d5692345a0b7cb5aeb8214f0f0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:08.070752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08bcf3d5692345a0b7cb5aeb8214f0f0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:08.093394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08bcf3d5692345a0b7cb5aeb8214f0f0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:08.105535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08bcf3d5692345a0b7cb5aeb8214f0f0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:08.106962 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08bcf3d5692345a0b7cb5aeb8214f0f0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:08.118202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 935/1871] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:07 2022] DELETE /identity/v3/users/08bcf3d5692345a0b7cb5aeb8214f0f0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:08.121274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:08.123965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:08.200114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba63ee-eaf7-4d68-bff5-0bf5488a47ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:08.201845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba63ee-eaf7-4d68-bff5-0bf5488a47ff service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:08.220630 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 937/1872] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:08.249473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c1fa6-752e-45fa-aa25-3fddba891205 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:08.266114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3065f-3249-4465-b088-64c84c86f343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:08.266815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3065f-3249-4465-b088-64c84c86f343 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:08.267129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3065f-3249-4465-b088-64c84c86f343 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:08.267793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3065f-3249-4465-b088-64c84c86f343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:08.269788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c9b3065f-3249-4465-b088-64c84c86f343 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:08.422804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3065f-3249-4465-b088-64c84c86f343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:08.424338 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3065f-3249-4465-b088-64c84c86f343 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:08.441633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 936/1873] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:09.049536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba63ee-eaf7-4d68-bff5-0bf5488a47ff service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.051078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-71ba63ee-eaf7-4d68-bff5-0bf5488a47ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:09.074070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba63ee-eaf7-4d68-bff5-0bf5488a47ff service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.088732 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3065f-3249-4465-b088-64c84c86f343 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.090335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c9b3065f-3249-4465-b088-64c84c86f343 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:09.096625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:09.194941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:09.221763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.221763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.221763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.221763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3 admin admin] PATH_INFO: `/v3/projects/c827d44edbc943c2b9c8b3b8a5228ec9` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.222400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c827d44edbc943c2b9c8b3b8a5228ec9)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.223413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.243930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3065f-3249-4465-b088-64c84c86f343 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.265275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f619aa-079d-414c-ab5a-10cc842e8a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.265990 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f619aa-079d-414c-ab5a-10cc842e8a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.266266 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f619aa-079d-414c-ab5a-10cc842e8a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.266537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f619aa-079d-414c-ab5a-10cc842e8a90 admin admin] PATH_INFO: `/v3/users/b311c2f122714c909fca2a9fd48fcbc0` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.268559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f619aa-079d-414c-ab5a-10cc842e8a90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b311c2f122714c909fca2a9fd48fcbc0)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.270487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f619aa-079d-414c-ab5a-10cc842e8a90 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.323895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c827d44edbc943c2b9c8b3b8a5228ec9', 'request_id': 'req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:09.342069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c827d44edbc943c2b9c8b3b8a5228ec9', 'request_id': 'req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:09.357394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c827d44edbc943c2b9c8b3b8a5228ec9', 'request_id': 'req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:09.378815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c8f619aa-079d-414c-ab5a-10cc842e8a90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b311c2f122714c909fca2a9fd48fcbc0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:09.388553 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3 admin admin] The token cache is being invalidate because project c827d44edbc943c2b9c8b3b8a5228ec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:09.388553 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:09.389223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3cdcdea1-bd0b-4a81-a839-f2215501d2f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:09.400015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 938/1874] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:09 2022] DELETE /identity/v3/projects/c827d44edbc943c2b9c8b3b8a5228ec9 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:09.404432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1dbe-0cd2-4ad9-83ab-05ad73c0838b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.404636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1dbe-0cd2-4ad9-83ab-05ad73c0838b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.405006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1dbe-0cd2-4ad9-83ab-05ad73c0838b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.414347 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c8f619aa-079d-414c-ab5a-10cc842e8a90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b311c2f122714c909fca2a9fd48fcbc0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:09.428625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c8f619aa-079d-414c-ab5a-10cc842e8a90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b311c2f122714c909fca2a9fd48fcbc0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:09.445999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c8f619aa-079d-414c-ab5a-10cc842e8a90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b311c2f122714c909fca2a9fd48fcbc0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:09.446665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c8f619aa-079d-414c-ab5a-10cc842e8a90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b311c2f122714c909fca2a9fd48fcbc0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:09.449758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-befe1dbe-0cd2-4ad9-83ab-05ad73c0838b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:09.452254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-befe1dbe-0cd2-4ad9-83ab-05ad73c0838b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:09.463714 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 937/1875] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:09 2022] DELETE /identity/v3/users/b311c2f122714c909fca2a9fd48fcbc0 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:09.468767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c44c-b3d6-42aa-a244-5bcb25fe47c1 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.469179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c44c-b3d6-42aa-a244-5bcb25fe47c1 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.472597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c44c-b3d6-42aa-a244-5bcb25fe47c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.515392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b6f8c44c-b3d6-42aa-a244-5bcb25fe47c1 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:09.517673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b6f8c44c-b3d6-42aa-a244-5bcb25fe47c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:09.601284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 939/1876] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:09.609407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 938/1877] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:09.611068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852f702-a65a-4aaa-b1b9-2835aa4262f9 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.611356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852f702-a65a-4aaa-b1b9-2835aa4262f9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.611631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852f702-a65a-4aaa-b1b9-2835aa4262f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.623136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8c44c-b3d6-42aa-a244-5bcb25fe47c1 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.627469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b6f8c44c-b3d6-42aa-a244-5bcb25fe47c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:09.641936 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-3852f702-a65a-4aaa-b1b9-2835aa4262f9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:09.643361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3852f702-a65a-4aaa-b1b9-2835aa4262f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:09.658305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 940/1878] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:09.663855 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852f702-a65a-4aaa-b1b9-2835aa4262f9 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.674363 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:09.703310 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852f702-a65a-4aaa-b1b9-2835aa4262f9 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.713445 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30458a6-e577-4000-9325-258d9ee91da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.715785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30458a6-e577-4000-9325-258d9ee91da5 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.719075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30458a6-e577-4000-9325-258d9ee91da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.719424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30458a6-e577-4000-9325-258d9ee91da5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.720334 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30458a6-e577-4000-9325-258d9ee91da5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:09.720735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30458a6-e577-4000-9325-258d9ee91da5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.722714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30458a6-e577-4000-9325-258d9ee91da5 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.735387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8c44c-b3d6-42aa-a244-5bcb25fe47c1 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.740490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa426f-7d35-4d7d-b41d-11db8e94bd49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.741822 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa426f-7d35-4d7d-b41d-11db8e94bd49 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.742437 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa426f-7d35-4d7d-b41d-11db8e94bd49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.742797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa426f-7d35-4d7d-b41d-11db8e94bd49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.755506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 941/1879] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:09.761451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30458a6-e577-4000-9325-258d9ee91da5 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.761657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa426f-7d35-4d7d-b41d-11db8e94bd49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.765179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa426f-7d35-4d7d-b41d-11db8e94bd49 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.783141 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 939/1880] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:09.788438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30458a6-e577-4000-9325-258d9ee91da5 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.793128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c49c6-6b39-4ceb-a039-87febc791fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.793824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c49c6-6b39-4ceb-a039-87febc791fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.794175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c49c6-6b39-4ceb-a039-87febc791fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.794545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c49c6-6b39-4ceb-a039-87febc791fd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.795338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c49c6-6b39-4ceb-a039-87febc791fd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.797924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c49c6-6b39-4ceb-a039-87febc791fd7 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.841692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb2ac0-5c93-45f2-bc02-b7f4c916623d None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.842879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb2ac0-5c93-45f2-bc02-b7f4c916623d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.842879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb2ac0-5c93-45f2-bc02-b7f4c916623d None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.844371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 940/1881] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:17:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:17:09.870104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f983d9-5a85-402a-a2d1-2f27248a35e2 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.870391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f983d9-5a85-402a-a2d1-2f27248a35e2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.871045 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f983d9-5a85-402a-a2d1-2f27248a35e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.888703 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 942/1882] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:09 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:09.894513 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c49c6-6b39-4ceb-a039-87febc791fd7 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.907177 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e1f983d9-5a85-402a-a2d1-2f27248a35e2 None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:09.908500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e1f983d9-5a85-402a-a2d1-2f27248a35e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:10.075976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:10.099014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 941/1883] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:17:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:10.118835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f983d9-5a85-402a-a2d1-2f27248a35e2 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:10.119453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c49c6-6b39-4ceb-a039-87febc791fd7 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:10.124046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e1f983d9-5a85-402a-a2d1-2f27248a35e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:10.138182 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076d3b6-e246-44c5-840a-df5a72e9dd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:10.138822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076d3b6-e246-44c5-840a-df5a72e9dd80 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.139382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076d3b6-e246-44c5-840a-df5a72e9dd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.139664 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076d3b6-e246-44c5-840a-df5a72e9dd80 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.141722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076d3b6-e246-44c5-840a-df5a72e9dd80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:10.143665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076d3b6-e246-44c5-840a-df5a72e9dd80 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:10.157035 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f983d9-5a85-402a-a2d1-2f27248a35e2 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:10.172791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9b15f-3b0b-4f23-a7c5-ea20a779ea8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:10.173550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9b15f-3b0b-4f23-a7c5-ea20a779ea8e service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.174133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9b15f-3b0b-4f23-a7c5-ea20a779ea8e service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.174133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9b15f-3b0b-4f23-a7c5-ea20a779ea8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.203139 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 943/1884] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:09 2022] POST /identity/v3/users => generated 295 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:10.203308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9b15f-3b0b-4f23-a7c5-ea20a779ea8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:10.205328 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076d3b6-e246-44c5-840a-df5a72e9dd80 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:10.211195 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9b15f-3b0b-4f23-a7c5-ea20a779ea8e service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:10.229847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076d3b6-e246-44c5-840a-df5a72e9dd80 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:10.242448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 942/1885] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:10.257748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a88826-bf1c-4893-8bed-f2f7a485bc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:10.259746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88826-bf1c-4893-8bed-f2f7a485bc73 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.259746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88826-bf1c-4893-8bed-f2f7a485bc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.259952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88826-bf1c-4893-8bed-f2f7a485bc73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.260662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a88826-bf1c-4893-8bed-f2f7a485bc73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:10.266253 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a88826-bf1c-4893-8bed-f2f7a485bc73 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:10.291815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9b15f-3b0b-4f23-a7c5-ea20a779ea8e service nova] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:10.305490 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 944/1886] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:10.306280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9b15f-3b0b-4f23-a7c5-ea20a779ea8e service nova] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:10.338684 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbf316-020b-4288-947f-90b567d51fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:10.339339 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbf316-020b-4288-947f-90b567d51fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.339644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbf316-020b-4288-947f-90b567d51fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.339892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbf316-020b-4288-947f-90b567d51fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.340584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbf316-020b-4288-947f-90b567d51fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:10.349087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbf316-020b-4288-947f-90b567d51fe1 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:10.379587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 943/1887] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:10.384727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a88826-bf1c-4893-8bed-f2f7a485bc73 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:10.402587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a88826-bf1c-4893-8bed-f2f7a485bc73 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:10.411779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3342b3-99f7-4f3b-9723-92a7750fe351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:10.425102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3342b3-99f7-4f3b-9723-92a7750fe351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.425434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3342b3-99f7-4f3b-9723-92a7750fe351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.425717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3342b3-99f7-4f3b-9723-92a7750fe351 admin admin] PATH_INFO: `/v3/projects/bb6d0cc47b424c719333114353ab6d03/users/ee567c5bd1d646d48b08af83bdca7039/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.470921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3342b3-99f7-4f3b-9723-92a7750fe351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6d0cc47b424c719333114353ab6d03, user_id=ee567c5bd1d646d48b08af83bdca7039, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:10.474027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3342b3-99f7-4f3b-9723-92a7750fe351 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:10.509991 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 945/1888] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:10 2022] PUT /identity/v3/projects/bb6d0cc47b424c719333114353ab6d03/users/ee567c5bd1d646d48b08af83bdca7039/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:10.522695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4aebbf-8449-4739-9f14-ff8f197c1e79 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.522985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4aebbf-8449-4739-9f14-ff8f197c1e79 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.523264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4aebbf-8449-4739-9f14-ff8f197c1e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.600761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-4f4aebbf-8449-4739-9f14-ff8f197c1e79 None None] MFA Rules not processed for user `ee567c5bd1d646d48b08af83bdca7039`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:10.602129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4f4aebbf-8449-4739-9f14-ff8f197c1e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:10.608894 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3342b3-99f7-4f3b-9723-92a7750fe351 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:10.610784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-be3342b3-99f7-4f3b-9723-92a7750fe351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:10.759488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 944/1889] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:10.760884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4aebbf-8449-4739-9f14-ff8f197c1e79 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:10.777512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4aebbf-8449-4739-9f14-ff8f197c1e79 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:10.785015 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173597b-be5f-4d26-a13c-fa3bb5d6dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:10.785839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173597b-be5f-4d26-a13c-fa3bb5d6dc70 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.786422 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173597b-be5f-4d26-a13c-fa3bb5d6dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.786422 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173597b-be5f-4d26-a13c-fa3bb5d6dc70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.787040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173597b-be5f-4d26-a13c-fa3bb5d6dc70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:10.789804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3342b3-99f7-4f3b-9723-92a7750fe351 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:10.790398 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173597b-be5f-4d26-a13c-fa3bb5d6dc70 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:10.804766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:10.805610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.805896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.806172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] PATH_INFO: `/v3/projects/76ac05546fa44c1f8aff5f3036fc1d47` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.808621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ac05546fa44c1f8aff5f3036fc1d47)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:10.814660 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:10.854360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:10.881034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 945/1890] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:10 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:10.884026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173597b-be5f-4d26-a13c-fa3bb5d6dc70 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:10.942382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ac05546fa44c1f8aff5f3036fc1d47', 'request_id': 'req-4dfb6c89-f041-4e42-a6d9-e6fdead94348'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:10.953028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ac05546fa44c1f8aff5f3036fc1d47', 'request_id': 'req-4dfb6c89-f041-4e42-a6d9-e6fdead94348'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:10.970793 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ac05546fa44c1f8aff5f3036fc1d47', 'request_id': 'req-4dfb6c89-f041-4e42-a6d9-e6fdead94348'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.004180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] The token cache is being invalidate because project 76ac05546fa44c1f8aff5f3036fc1d47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:11.004681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.010943 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.019873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 946/1891] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:10 2022] DELETE /identity/v3/projects/76ac05546fa44c1f8aff5f3036fc1d47 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:11.033848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:11.041770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:11.079170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173597b-be5f-4d26-a13c-fa3bb5d6dc70 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:11.133070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:11.202857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb6c89-f041-4e42-a6d9-e6fdead94348 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:11.211367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:11.212873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:11.213407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:11.213929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] PATH_INFO: `/v3/users/ec903c7876204887b30ce211ac243b8a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:11.216620 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec903c7876204887b30ce211ac243b8a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:11.220328 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:11.247788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9dbe13-8a51-42fc-90cf-30b7ebb8f047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:11.248940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9dbe13-8a51-42fc-90cf-30b7ebb8f047 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:11.248940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9dbe13-8a51-42fc-90cf-30b7ebb8f047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:11.249627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9dbe13-8a51-42fc-90cf-30b7ebb8f047 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:11.250474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9dbe13-8a51-42fc-90cf-30b7ebb8f047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:11.254383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9dbe13-8a51-42fc-90cf-30b7ebb8f047 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:11.293950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 946/1892] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:10 2022] POST /identity/v3/users => generated 300 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:11.298481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9dbe13-8a51-42fc-90cf-30b7ebb8f047 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:11.301716 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ac9dbe13-8a51-42fc-90cf-30b7ebb8f047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:11.337330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec903c7876204887b30ce211ac243b8a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.353909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec903c7876204887b30ce211ac243b8a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.355216 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec903c7876204887b30ce211ac243b8a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.356661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec903c7876204887b30ce211ac243b8a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.357918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9dbe13-8a51-42fc-90cf-30b7ebb8f047 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:11.372579 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa61d2-18c6-4a87-8054-44f8dd0359d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:11.373724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa61d2-18c6-4a87-8054-44f8dd0359d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:11.374174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa61d2-18c6-4a87-8054-44f8dd0359d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:11.374661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa61d2-18c6-4a87-8054-44f8dd0359d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:11.375774 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa61d2-18c6-4a87-8054-44f8dd0359d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:11.379007 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa61d2-18c6-4a87-8054-44f8dd0359d3 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:11.386894 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec903c7876204887b30ce211ac243b8a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.400741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 947/1893] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:11.405211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa61d2-18c6-4a87-8054-44f8dd0359d3 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:11.409004 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7aaa61d2-18c6-4a87-8054-44f8dd0359d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:11.421838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 947/1894] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:11 2022] DELETE /identity/v3/users/ec903c7876204887b30ce211ac243b8a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:11.439078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa61d2-18c6-4a87-8054-44f8dd0359d3 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:11.542532 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c200b-be0d-4689-96c0-7b7256426aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:11.544498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c200b-be0d-4689-96c0-7b7256426aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:11.544947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c200b-be0d-4689-96c0-7b7256426aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:11.545225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c200b-be0d-4689-96c0-7b7256426aa7 admin admin] PATH_INFO: `/v3/projects/af2464a5c1874cd197699fac74ac3a1d/users/d63e1b4332654fc98f4c2860057f7c51/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:11.570642 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c200b-be0d-4689-96c0-7b7256426aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2464a5c1874cd197699fac74ac3a1d, user_id=d63e1b4332654fc98f4c2860057f7c51, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:11.582246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c200b-be0d-4689-96c0-7b7256426aa7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:11.610943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 948/1895] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:11 2022] PUT /identity/v3/projects/af2464a5c1874cd197699fac74ac3a1d/users/d63e1b4332654fc98f4c2860057f7c51/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:11.615271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:11.630735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605436a-9450-4bd5-a791-6110bc62edcc admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:11.751028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bce77-a2bb-4cd6-85b4-bdfd8339ecfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:11.751900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bce77-a2bb-4cd6-85b4-bdfd8339ecfe admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:11.753190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bce77-a2bb-4cd6-85b4-bdfd8339ecfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:11.753553 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bce77-a2bb-4cd6-85b4-bdfd8339ecfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:11.754377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bce77-a2bb-4cd6-85b4-bdfd8339ecfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:11.756594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bce77-a2bb-4cd6-85b4-bdfd8339ecfe admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:11.770707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 948/1896] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:11.775636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c200b-be0d-4689-96c0-7b7256426aa7 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:11.788496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c200b-be0d-4689-96c0-7b7256426aa7 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:11.800360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c56cb9-f7dd-4d44-8e6a-0480d0888487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:11.801263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c56cb9-f7dd-4d44-8e6a-0480d0888487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:11.801685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c56cb9-f7dd-4d44-8e6a-0480d0888487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:11.802118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c56cb9-f7dd-4d44-8e6a-0480d0888487 admin admin] PATH_INFO: `/v3/projects/af2464a5c1874cd197699fac74ac3a1d/users/d63e1b4332654fc98f4c2860057f7c51/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:11.813767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c56cb9-f7dd-4d44-8e6a-0480d0888487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2464a5c1874cd197699fac74ac3a1d, user_id=d63e1b4332654fc98f4c2860057f7c51, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:11.819842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c56cb9-f7dd-4d44-8e6a-0480d0888487 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:11.825342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:11.842151 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 949/1897] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:11 2022] PUT /identity/v3/projects/af2464a5c1874cd197699fac74ac3a1d/users/d63e1b4332654fc98f4c2860057f7c51/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:11.854033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7ac34-35aa-445d-a11c-8f79007a753c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:11.854339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7ac34-35aa-445d-a11c-8f79007a753c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:11.854662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7ac34-35aa-445d-a11c-8f79007a753c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:11.956338 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c56cb9-f7dd-4d44-8e6a-0480d0888487 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:11.958382 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b4c56cb9-f7dd-4d44-8e6a-0480d0888487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:11.976939 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-b8b7ac34-35aa-445d-a11c-8f79007a753c None None] MFA Rules not processed for user `d63e1b4332654fc98f4c2860057f7c51`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:11.978353 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b8b7ac34-35aa-445d-a11c-8f79007a753c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.045567 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:12.106640 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 949/1898] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:12.120961 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7ac34-35aa-445d-a11c-8f79007a753c None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.123247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b8b7ac34-35aa-445d-a11c-8f79007a753c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.124930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c56cb9-f7dd-4d44-8e6a-0480d0888487 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.139604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53654c76-dde3-472f-a6f1-3bebe08fe807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.140538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53654c76-dde3-472f-a6f1-3bebe08fe807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.140810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53654c76-dde3-472f-a6f1-3bebe08fe807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.141085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53654c76-dde3-472f-a6f1-3bebe08fe807 admin admin] PATH_INFO: `/v3/projects/d1208c94753f4960aaaea57f880cd991` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.142232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53654c76-dde3-472f-a6f1-3bebe08fe807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1208c94753f4960aaaea57f880cd991)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.144173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53654c76-dde3-472f-a6f1-3bebe08fe807 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:12.208651 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-53654c76-dde3-472f-a6f1-3bebe08fe807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1208c94753f4960aaaea57f880cd991', 'request_id': 'req-53654c76-dde3-472f-a6f1-3bebe08fe807'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.213760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7ac34-35aa-445d-a11c-8f79007a753c None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.218768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-53654c76-dde3-472f-a6f1-3bebe08fe807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1208c94753f4960aaaea57f880cd991', 'request_id': 'req-53654c76-dde3-472f-a6f1-3bebe08fe807'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.230695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-53654c76-dde3-472f-a6f1-3bebe08fe807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1208c94753f4960aaaea57f880cd991', 'request_id': 'req-53654c76-dde3-472f-a6f1-3bebe08fe807'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.247277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-53654c76-dde3-472f-a6f1-3bebe08fe807 admin admin] The token cache is being invalidate because project d1208c94753f4960aaaea57f880cd991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:12.247650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-53654c76-dde3-472f-a6f1-3bebe08fe807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.248635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-53654c76-dde3-472f-a6f1-3bebe08fe807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.251040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 950/1899] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:11 2022] DELETE /identity/v3/projects/d1208c94753f4960aaaea57f880cd991 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:12.256739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59232b15-ffad-40c2-bc9f-ba476bdf042b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.257095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59232b15-ffad-40c2-bc9f-ba476bdf042b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.257439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59232b15-ffad-40c2-bc9f-ba476bdf042b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.302362 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f83e2-8acb-44fa-9306-c9529ca6b64a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.302655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-59232b15-ffad-40c2-bc9f-ba476bdf042b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:12.303541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f83e2-8acb-44fa-9306-c9529ca6b64a service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.303813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f83e2-8acb-44fa-9306-c9529ca6b64a service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.304114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f83e2-8acb-44fa-9306-c9529ca6b64a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.305587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-59232b15-ffad-40c2-bc9f-ba476bdf042b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.305932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e29f83e2-8acb-44fa-9306-c9529ca6b64a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.393876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f83e2-8acb-44fa-9306-c9529ca6b64a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.395324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f83e2-8acb-44fa-9306-c9529ca6b64a service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:12.408561 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 950/1900] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:12.409463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f83e2-8acb-44fa-9306-c9529ca6b64a service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.410952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e29f83e2-8acb-44fa-9306-c9529ca6b64a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.417801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 951/1901] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:12.428633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e5f7a-ca72-44cc-9e2c-eedd6fb13b09 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.433705 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f83e2-8acb-44fa-9306-c9529ca6b64a service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.435086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e5f7a-ca72-44cc-9e2c-eedd6fb13b09 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.435871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e5f7a-ca72-44cc-9e2c-eedd6fb13b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.439162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6211cd3-42ea-48c2-98f9-6f902cef9e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.439905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6211cd3-42ea-48c2-98f9-6f902cef9e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.440214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6211cd3-42ea-48c2-98f9-6f902cef9e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.440553 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6211cd3-42ea-48c2-98f9-6f902cef9e1e admin admin] PATH_INFO: `/v3/users/8840c29d52bf4aba946243e205b75e23` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.441764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6211cd3-42ea-48c2-98f9-6f902cef9e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8840c29d52bf4aba946243e205b75e23)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.444016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6211cd3-42ea-48c2-98f9-6f902cef9e1e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:12.468401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b08e5f7a-ca72-44cc-9e2c-eedd6fb13b09 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:12.469057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b08e5f7a-ca72-44cc-9e2c-eedd6fb13b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.477676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 952/1902] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:12.481698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e5f7a-ca72-44cc-9e2c-eedd6fb13b09 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.494598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e5f7a-ca72-44cc-9e2c-eedd6fb13b09 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.501596 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:12.505052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b95b0-8f51-4a57-8a12-3db8a0803d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.505824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b95b0-8f51-4a57-8a12-3db8a0803d42 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.506392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b95b0-8f51-4a57-8a12-3db8a0803d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.507050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b95b0-8f51-4a57-8a12-3db8a0803d42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.507957 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b95b0-8f51-4a57-8a12-3db8a0803d42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:12.508882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b95b0-8f51-4a57-8a12-3db8a0803d42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.512344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b95b0-8f51-4a57-8a12-3db8a0803d42 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:12.520367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e6211cd3-42ea-48c2-98f9-6f902cef9e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8840c29d52bf4aba946243e205b75e23'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.530938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 953/1903] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:17:12.535101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b95b0-8f51-4a57-8a12-3db8a0803d42 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.540495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e6211cd3-42ea-48c2-98f9-6f902cef9e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8840c29d52bf4aba946243e205b75e23'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.541229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e6211cd3-42ea-48c2-98f9-6f902cef9e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8840c29d52bf4aba946243e205b75e23'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.542155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e6211cd3-42ea-48c2-98f9-6f902cef9e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8840c29d52bf4aba946243e205b75e23'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.546824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b95b0-8f51-4a57-8a12-3db8a0803d42 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.552688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b0bac-780b-493b-993c-c87b8600a92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.553358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b0bac-780b-493b-993c-c87b8600a92b admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.553634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b0bac-780b-493b-993c-c87b8600a92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.553908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b0bac-780b-493b-993c-c87b8600a92b admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.554975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b0bac-780b-493b-993c-c87b8600a92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.560103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b0bac-780b-493b-993c-c87b8600a92b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:12.560332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e6211cd3-42ea-48c2-98f9-6f902cef9e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8840c29d52bf4aba946243e205b75e23'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.565287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:12.577679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 951/1904] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:12 2022] DELETE /identity/v3/users/8840c29d52bf4aba946243e205b75e23 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:12.582494 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f0e96-8f10-4548-829d-28f9eef9e654 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.582846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f0e96-8f10-4548-829d-28f9eef9e654 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.583321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f0e96-8f10-4548-829d-28f9eef9e654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.609372 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c33f0e96-8f10-4548-829d-28f9eef9e654 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:12.612198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 954/1905] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:12 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:12.612342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c33f0e96-8f10-4548-829d-28f9eef9e654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.612424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b0bac-780b-493b-993c-c87b8600a92b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.615095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8f3b0bac-780b-493b-993c-c87b8600a92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.684448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b0bac-780b-493b-993c-c87b8600a92b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.689327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 952/1906] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:12.690170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f0e96-8f10-4548-829d-28f9eef9e654 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.691628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c33f0e96-8f10-4548-829d-28f9eef9e654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.698648 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.699589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.699930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.700240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] PATH_INFO: `/v3/users/ee567c5bd1d646d48b08af83bdca7039` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.702152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee567c5bd1d646d48b08af83bdca7039)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.704318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:12.704726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f0e96-8f10-4548-829d-28f9eef9e654 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.711098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdcad0-db87-4e67-be6f-0ff58a78f15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.711725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdcad0-db87-4e67-be6f-0ff58a78f15c admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.712025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdcad0-db87-4e67-be6f-0ff58a78f15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.712314 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdcad0-db87-4e67-be6f-0ff58a78f15c admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.712956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdcad0-db87-4e67-be6f-0ff58a78f15c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.714922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdcad0-db87-4e67-be6f-0ff58a78f15c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:12.742036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 953/1907] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:12 2022] POST /identity/v3/users => generated 292 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:12.745405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdcad0-db87-4e67-be6f-0ff58a78f15c admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.745405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-11cdcad0-db87-4e67-be6f-0ff58a78f15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.775899 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee567c5bd1d646d48b08af83bdca7039'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.789833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee567c5bd1d646d48b08af83bdca7039'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.797206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee567c5bd1d646d48b08af83bdca7039'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.805429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee567c5bd1d646d48b08af83bdca7039'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.806294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee567c5bd1d646d48b08af83bdca7039'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.812348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdcad0-db87-4e67-be6f-0ff58a78f15c admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.812945 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 955/1908] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:12 2022] DELETE /identity/v3/users/ee567c5bd1d646d48b08af83bdca7039 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:12.813798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.815164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.860412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:12.866355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2713f3-1712-43e9-b6d6-b6d627cb33bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.867122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2713f3-1712-43e9-b6d6-b6d627cb33bc service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.867448 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2713f3-1712-43e9-b6d6-b6d627cb33bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.867709 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2713f3-1712-43e9-b6d6-b6d627cb33bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.870680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ff2713f3-1712-43e9-b6d6-b6d627cb33bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.889354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fe5e5-5dfa-4379-bcb9-20c81bcba088 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.894641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddb8a4-3258-41ec-8741-9f605ac658b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.895279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddb8a4-3258-41ec-8741-9f605ac658b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.895549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddb8a4-3258-41ec-8741-9f605ac658b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.895818 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddb8a4-3258-41ec-8741-9f605ac658b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.897174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddb8a4-3258-41ec-8741-9f605ac658b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.898995 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddb8a4-3258-41ec-8741-9f605ac658b5 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:12.904927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:12.910509 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 956/1909] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:12.913622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f9dd3-567e-4257-b3bf-01c4b654ffe9 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.913902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f9dd3-567e-4257-b3bf-01c4b654ffe9 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.919566 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f9dd3-567e-4257-b3bf-01c4b654ffe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.942219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e46f9dd3-567e-4257-b3bf-01c4b654ffe9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:12.944282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e46f9dd3-567e-4257-b3bf-01c4b654ffe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.947902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2713f3-1712-43e9-b6d6-b6d627cb33bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.949753 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2713f3-1712-43e9-b6d6-b6d627cb33bc service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:12.966155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 957/1910] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:12.966155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f9dd3-567e-4257-b3bf-01c4b654ffe9 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.966827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 954/1911] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:12.973096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2713f3-1712-43e9-b6d6-b6d627cb33bc service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.975032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ff2713f3-1712-43e9-b6d6-b6d627cb33bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.985523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f9dd3-567e-4257-b3bf-01c4b654ffe9 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.987021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2713f3-1712-43e9-b6d6-b6d627cb33bc service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.991985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0df15-9636-419a-b364-16ed5bda8ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.992441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070059d-df3f-4308-9400-914dd6edaaf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.992953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0df15-9636-419a-b364-16ed5bda8ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.993069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070059d-df3f-4308-9400-914dd6edaaf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.993335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070059d-df3f-4308-9400-914dd6edaaf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.993439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0df15-9636-419a-b364-16ed5bda8ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.993608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070059d-df3f-4308-9400-914dd6edaaf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.994097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0df15-9636-419a-b364-16ed5bda8ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.994926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0df15-9636-419a-b364-16ed5bda8ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.997297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0df15-9636-419a-b364-16ed5bda8ce7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:13.008540 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070059d-df3f-4308-9400-914dd6edaaf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:13.010106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070059d-df3f-4308-9400-914dd6edaaf6 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:13.012035 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 958/1912] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:13.018027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0df15-9636-419a-b364-16ed5bda8ce7 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:13.027820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 955/1913] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:13.031862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0df15-9636-419a-b364-16ed5bda8ce7 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:13.038515 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c939e-0717-4d37-9549-303553f7c0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:13.039613 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c939e-0717-4d37-9549-303553f7c0cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:13.040101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c939e-0717-4d37-9549-303553f7c0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:13.041424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c939e-0717-4d37-9549-303553f7c0cd admin admin] PATH_INFO: `/v3/projects/d485e5cd429d444f9435a12f001d8109/users/93d9aad7dd1f43e0ab264ab5157ae845/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:13.082834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c939e-0717-4d37-9549-303553f7c0cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d485e5cd429d444f9435a12f001d8109, user_id=93d9aad7dd1f43e0ab264ab5157ae845, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:13.086795 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c939e-0717-4d37-9549-303553f7c0cd admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:13.113475 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 959/1914] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:13 2022] PUT /identity/v3/projects/d485e5cd429d444f9435a12f001d8109/users/93d9aad7dd1f43e0ab264ab5157ae845/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:13.134094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54310bec-0139-4d74-a3da-73f002b38b56 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:13.134420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54310bec-0139-4d74-a3da-73f002b38b56 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:13.134818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54310bec-0139-4d74-a3da-73f002b38b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:13.171936 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-54310bec-0139-4d74-a3da-73f002b38b56 None None] MFA Rules not processed for user `93d9aad7dd1f43e0ab264ab5157ae845`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:13.173967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-54310bec-0139-4d74-a3da-73f002b38b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:13.364648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 956/1915] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:13.366813 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c939e-0717-4d37-9549-303553f7c0cd admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:13.501701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c939e-0717-4d37-9549-303553f7c0cd admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:13.514070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54310bec-0139-4d74-a3da-73f002b38b56 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:13.518023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-54310bec-0139-4d74-a3da-73f002b38b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:13.519155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34bc62-8a5d-4a09-8848-13919364d67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:13.520568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34bc62-8a5d-4a09-8848-13919364d67c admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:13.521156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34bc62-8a5d-4a09-8848-13919364d67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:13.521898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34bc62-8a5d-4a09-8848-13919364d67c admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:13.523309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34bc62-8a5d-4a09-8848-13919364d67c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:13.527912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34bc62-8a5d-4a09-8848-13919364d67c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:13.560208 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54310bec-0139-4d74-a3da-73f002b38b56 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:13.576954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:13.577634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:13.578004 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:13.578550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] PATH_INFO: `/v3/projects/760804bf13744adeb30438d7c75a5a8e` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:13.580043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=760804bf13744adeb30438d7c75a5a8e)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:13.582097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:13.587265 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 960/1916] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:13 2022] POST /identity/v3/projects => generated 383 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:13.593594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34bc62-8a5d-4a09-8848-13919364d67c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:13.690904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '760804bf13744adeb30438d7c75a5a8e', 'request_id': 'req-5041cd60-0d5d-4eef-a480-8e4171597a4e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:13.704402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '760804bf13744adeb30438d7c75a5a8e', 'request_id': 'req-5041cd60-0d5d-4eef-a480-8e4171597a4e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:13.715775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '760804bf13744adeb30438d7c75a5a8e', 'request_id': 'req-5041cd60-0d5d-4eef-a480-8e4171597a4e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:13.729000 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:13.734395 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] The token cache is being invalidate because project 760804bf13744adeb30438d7c75a5a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:13.735385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:13.736921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:13.740063 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 957/1917] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:13 2022] DELETE /identity/v3/projects/760804bf13744adeb30438d7c75a5a8e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:13.745614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:13.749604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:13.749980 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34bc62-8a5d-4a09-8848-13919364d67c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:13.844939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:13.860487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d4985-2036-4ea1-96ef-76633076d015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:13.861124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d4985-2036-4ea1-96ef-76633076d015 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:13.861728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d4985-2036-4ea1-96ef-76633076d015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:13.862146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d4985-2036-4ea1-96ef-76633076d015 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:13.862856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d4985-2036-4ea1-96ef-76633076d015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:13.866629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d4985-2036-4ea1-96ef-76633076d015 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:13.902130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041cd60-0d5d-4eef-a480-8e4171597a4e admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:13.908392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:13.908662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 961/1918] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:13 2022] POST /identity/v3/users => generated 298 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:13.908985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:13.909265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:13.909539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] PATH_INFO: `/v3/users/55df8ee8d69d482395c7582299d75285` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:13.910376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d4985-2036-4ea1-96ef-76633076d015 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:13.910750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55df8ee8d69d482395c7582299d75285)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:13.912865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ca6d4985-2036-4ea1-96ef-76633076d015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:13.913085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:13.926255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d4985-2036-4ea1-96ef-76633076d015 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:13.933006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:13.934071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:13.934308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:13.934568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] PATH_INFO: `/v3/projects/bb6d0cc47b424c719333114353ab6d03` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:13.935817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb6d0cc47b424c719333114353ab6d03)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:13.938211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:13.980145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55df8ee8d69d482395c7582299d75285'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:13.990952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55df8ee8d69d482395c7582299d75285'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:13.992383 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55df8ee8d69d482395c7582299d75285'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:13.993014 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55df8ee8d69d482395c7582299d75285'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:13.997069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb6d0cc47b424c719333114353ab6d03', 'request_id': 'req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.008281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb6d0cc47b424c719333114353ab6d03', 'request_id': 'req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.011456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55df8ee8d69d482395c7582299d75285'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.016977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb6d0cc47b424c719333114353ab6d03', 'request_id': 'req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.032969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 958/1919] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:13 2022] DELETE /identity/v3/users/55df8ee8d69d482395c7582299d75285 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:14.034662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:14.036898 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:14.043031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] The token cache is being invalidate because project bb6d0cc47b424c719333114353ab6d03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:14.043551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.044940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.048602 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 962/1920] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:13 2022] DELETE /identity/v3/projects/bb6d0cc47b424c719333114353ab6d03 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:14.055260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:14.057632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:14.131698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901a546-b23f-40ca-8a3c-1345470bf3b7 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:14.137921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d1646-c8be-4025-a16c-e53d30046029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:14.138557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d1646-c8be-4025-a16c-e53d30046029 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.138836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d1646-c8be-4025-a16c-e53d30046029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.139100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d1646-c8be-4025-a16c-e53d30046029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.140037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d1646-c8be-4025-a16c-e53d30046029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:14.141656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d1646-c8be-4025-a16c-e53d30046029 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:14.156392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 959/1921] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:14.157919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d1646-c8be-4025-a16c-e53d30046029 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:14.160061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-dd7d1646-c8be-4025-a16c-e53d30046029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:14.175729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d1646-c8be-4025-a16c-e53d30046029 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:14.175983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e08a5d-8b0c-4ec4-a216-b7dbf1f80822 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:14.185535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82a15c-01cd-4b57-b2e1-3b1cf0823c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:14.185828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:14.185994 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82a15c-01cd-4b57-b2e1-3b1cf0823c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.186138 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.186282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82a15c-01cd-4b57-b2e1-3b1cf0823c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.186282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82a15c-01cd-4b57-b2e1-3b1cf0823c27 admin admin] PATH_INFO: `/v3/projects/db52a8ec32e148068c370876e8d3af28/users/d70a6b721dfc4cea8990c3ac45942576/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.186558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.186558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] PATH_INFO: `/v3/users/d63e1b4332654fc98f4c2860057f7c51` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.188755 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=d63e1b4332654fc98f4c2860057f7c51)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:14.193083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:14.208335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:14.228544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82a15c-01cd-4b57-b2e1-3b1cf0823c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db52a8ec32e148068c370876e8d3af28, user_id=d70a6b721dfc4cea8990c3ac45942576, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:14.236532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82a15c-01cd-4b57-b2e1-3b1cf0823c27 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:14.267969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 960/1922] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:14 2022] PUT /identity/v3/projects/db52a8ec32e148068c370876e8d3af28/users/d70a6b721dfc4cea8990c3ac45942576/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:14.276108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82a15c-01cd-4b57-b2e1-3b1cf0823c27 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:14.353230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd63e1b4332654fc98f4c2860057f7c51'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.377292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd63e1b4332654fc98f4c2860057f7c51'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.378685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=80,keystone:INSERT=4,keystone:DELETE=16 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:14.395923 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd63e1b4332654fc98f4c2860057f7c51'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.398097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82a15c-01cd-4b57-b2e1-3b1cf0823c27 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:14.404285 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726161f5-401f-4608-b452-3dd9fb2db038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:14.405478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726161f5-401f-4608-b452-3dd9fb2db038 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.406200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726161f5-401f-4608-b452-3dd9fb2db038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.406876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726161f5-401f-4608-b452-3dd9fb2db038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.407845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726161f5-401f-4608-b452-3dd9fb2db038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:14.410133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726161f5-401f-4608-b452-3dd9fb2db038 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:14.414462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd63e1b4332654fc98f4c2860057f7c51'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.416621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd63e1b4332654fc98f4c2860057f7c51'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.426092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 963/1923] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:14 2022] DELETE /identity/v3/users/d63e1b4332654fc98f4c2860057f7c51 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:14.430551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 961/1924] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:14.431934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:14.433627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:14.535745 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726161f5-401f-4608-b452-3dd9fb2db038 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:14.542398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-726161f5-401f-4608-b452-3dd9fb2db038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:14.555552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e85604-dc35-4397-8786-66e9f65bf7ca admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:14.563922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dae34-a7a4-493b-922d-e7960656bf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:14.564346 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dae34-a7a4-493b-922d-e7960656bf96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.564798 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dae34-a7a4-493b-922d-e7960656bf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.565212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dae34-a7a4-493b-922d-e7960656bf96 admin admin] PATH_INFO: `/v3/projects/db52a8ec32e148068c370876e8d3af28/users/d70a6b721dfc4cea8990c3ac45942576/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.569109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dae34-a7a4-493b-922d-e7960656bf96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db52a8ec32e148068c370876e8d3af28, user_id=d70a6b721dfc4cea8990c3ac45942576, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:14.572402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726161f5-401f-4608-b452-3dd9fb2db038 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:14.574941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dae34-a7a4-493b-922d-e7960656bf96 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:14.579484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:14.580364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.580650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.581067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74 admin admin] PATH_INFO: `/v3/projects/e6330b3eeabf45cb83066cd401e24460` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.582441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6330b3eeabf45cb83066cd401e24460)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:14.585174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:14.606087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 964/1925] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:14 2022] PUT /identity/v3/projects/db52a8ec32e148068c370876e8d3af28/users/d70a6b721dfc4cea8990c3ac45942576/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:14.623324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16486119-ca2a-4395-83ee-dc07bf1a3105 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.623324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16486119-ca2a-4395-83ee-dc07bf1a3105 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.623790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16486119-ca2a-4395-83ee-dc07bf1a3105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.679591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6330b3eeabf45cb83066cd401e24460', 'request_id': 'req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.705023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6330b3eeabf45cb83066cd401e24460', 'request_id': 'req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.719515 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6330b3eeabf45cb83066cd401e24460', 'request_id': 'req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.750044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74 admin admin] The token cache is being invalidate because project e6330b3eeabf45cb83066cd401e24460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:14.751032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.752662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-31cdee02-1e98-4f1c-9f4f-60baec2e1d74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.763161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 962/1926] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:14 2022] DELETE /identity/v3/projects/e6330b3eeabf45cb83066cd401e24460 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:14.768513 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2ec61-6203-416e-809f-41bdd92177af None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.769092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2ec61-6203-416e-809f-41bdd92177af None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.769704 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2ec61-6203-416e-809f-41bdd92177af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.773941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-16486119-ca2a-4395-83ee-dc07bf1a3105 None None] MFA Rules not processed for user `d70a6b721dfc4cea8990c3ac45942576`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:14.782453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-16486119-ca2a-4395-83ee-dc07bf1a3105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:14.829676 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-86a2ec61-6203-416e-809f-41bdd92177af None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:14.832807 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-86a2ec61-6203-416e-809f-41bdd92177af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:14.961847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 963/1927] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:14.963850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 965/1928] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:14.971982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c9867-a878-4cdd-bd28-01b3ff7bf0ac None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.972569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c9867-a878-4cdd-bd28-01b3ff7bf0ac None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.973148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c9867-a878-4cdd-bd28-01b3ff7bf0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.985883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16486119-ca2a-4395-83ee-dc07bf1a3105 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:14.988281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-16486119-ca2a-4395-83ee-dc07bf1a3105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:15.053797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-b61c9867-a878-4cdd-bd28-01b3ff7bf0ac None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:15.061815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b61c9867-a878-4cdd-bd28-01b3ff7bf0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:15.091568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 964/1929] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:15.094368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c9867-a878-4cdd-bd28-01b3ff7bf0ac None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:15.105858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:15.114926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c9867-a878-4cdd-bd28-01b3ff7bf0ac None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:15.141442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe670a3-9389-44da-87ea-1ff08e7c83e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:15.142532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe670a3-9389-44da-87ea-1ff08e7c83e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.143214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe670a3-9389-44da-87ea-1ff08e7c83e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.144205 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe670a3-9389-44da-87ea-1ff08e7c83e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.145589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe670a3-9389-44da-87ea-1ff08e7c83e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:15.146241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe670a3-9389-44da-87ea-1ff08e7c83e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:15.151089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe670a3-9389-44da-87ea-1ff08e7c83e0 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:15.173535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16486119-ca2a-4395-83ee-dc07bf1a3105 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:15.180252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 965/1930] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:17:15.183137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e428f0-b103-4121-87ca-5c2a95f27e87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:15.184376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e428f0-b103-4121-87ca-5c2a95f27e87 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.184927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e428f0-b103-4121-87ca-5c2a95f27e87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.185835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e428f0-b103-4121-87ca-5c2a95f27e87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.185938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe670a3-9389-44da-87ea-1ff08e7c83e0 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:15.189970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e6e428f0-b103-4121-87ca-5c2a95f27e87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:15.210599 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe670a3-9389-44da-87ea-1ff08e7c83e0 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:15.214693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:15.222957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4d932-bd99-4da2-b6c2-bed039355120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:15.225727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4d932-bd99-4da2-b6c2-bed039355120 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.226687 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4d932-bd99-4da2-b6c2-bed039355120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.226687 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4d932-bd99-4da2-b6c2-bed039355120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.227896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4d932-bd99-4da2-b6c2-bed039355120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:15.232996 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4d932-bd99-4da2-b6c2-bed039355120 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:15.295464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 966/1931] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:15 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:15.298274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4d932-bd99-4da2-b6c2-bed039355120 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:15.301970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d5e4d932-bd99-4da2-b6c2-bed039355120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:15.343138 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e428f0-b103-4121-87ca-5c2a95f27e87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:15.345639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e428f0-b103-4121-87ca-5c2a95f27e87 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:15.364176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 966/1932] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:15.370288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e428f0-b103-4121-87ca-5c2a95f27e87 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:15.391770 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e428f0-b103-4121-87ca-5c2a95f27e87 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:15.470374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4d932-bd99-4da2-b6c2-bed039355120 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:15.476741 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:15.477375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.477686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.478995 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] PATH_INFO: `/v3/projects/af2464a5c1874cd197699fac74ac3a1d` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.481836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] RBAC: Authorizing `identity:delete_project(project_id=af2464a5c1874cd197699fac74ac3a1d)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:15.486030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:15.540698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e93d3-a597-4c65-bff1-ae03fef87548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:15.541857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e93d3-a597-4c65-bff1-ae03fef87548 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.542427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e93d3-a597-4c65-bff1-ae03fef87548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.542979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e93d3-a597-4c65-bff1-ae03fef87548 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.545414 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e93d3-a597-4c65-bff1-ae03fef87548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:15.552156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e93d3-a597-4c65-bff1-ae03fef87548 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:15.599984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 967/1933] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:15 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:15.605583 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e93d3-a597-4c65-bff1-ae03fef87548 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:15.623530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af2464a5c1874cd197699fac74ac3a1d', 'request_id': 'req-4ee3df78-3884-455c-947a-0a58acdeacda'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:15.627748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e93d3-a597-4c65-bff1-ae03fef87548 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:15.632467 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af2464a5c1874cd197699fac74ac3a1d', 'request_id': 'req-4ee3df78-3884-455c-947a-0a58acdeacda'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:15.643311 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af2464a5c1874cd197699fac74ac3a1d', 'request_id': 'req-4ee3df78-3884-455c-947a-0a58acdeacda'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:15.673847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] The token cache is being invalidate because project af2464a5c1874cd197699fac74ac3a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:15.674203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:15.675010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:15.686291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 967/1934] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:15 2022] DELETE /identity/v3/projects/af2464a5c1874cd197699fac74ac3a1d => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:15.687475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:15.689274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:15.754809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbc465-7122-4bdc-946f-3708d6f7d0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:15.755446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc465-7122-4bdc-946f-3708d6f7d0ef admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.756589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc465-7122-4bdc-946f-3708d6f7d0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.756965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc465-7122-4bdc-946f-3708d6f7d0ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.757654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbc465-7122-4bdc-946f-3708d6f7d0ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:15.759314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbc465-7122-4bdc-946f-3708d6f7d0ef admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:15.773009 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 968/1935] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:15.774544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b03dd-c004-45bc-a903-f9adb3641712 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.774669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b03dd-c004-45bc-a903-f9adb3641712 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.778668 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b03dd-c004-45bc-a903-f9adb3641712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.800323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b83b03dd-c004-45bc-a903-f9adb3641712 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:15.803365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b83b03dd-c004-45bc-a903-f9adb3641712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:15.817973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 969/1936] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:15.818589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:15.821867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b03dd-c004-45bc-a903-f9adb3641712 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:15.825954 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b83b03dd-c004-45bc-a903-f9adb3641712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:15.827439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3df78-3884-455c-947a-0a58acdeacda admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:15.841155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:15.841910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.842217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.843044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] PATH_INFO: `/v3/users/1c040a27bd92494db1fad3b7ae7fba45` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.843264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b03dd-c004-45bc-a903-f9adb3641712 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:15.845134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c040a27bd92494db1fad3b7ae7fba45)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:15.848751 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:15.864187 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150bdcf-4c1f-4f55-8a24-0629648504ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:15.864187 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150bdcf-4c1f-4f55-8a24-0629648504ab admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.864187 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150bdcf-4c1f-4f55-8a24-0629648504ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.864187 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150bdcf-4c1f-4f55-8a24-0629648504ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.864783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150bdcf-4c1f-4f55-8a24-0629648504ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:15.864783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150bdcf-4c1f-4f55-8a24-0629648504ab admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:15.876932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 970/1937] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:15.883506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174fdd6-a4fa-4de8-8c49-49e9caa29af6 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.884030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174fdd6-a4fa-4de8-8c49-49e9caa29af6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.884599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174fdd6-a4fa-4de8-8c49-49e9caa29af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.916030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c040a27bd92494db1fad3b7ae7fba45'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:15.921457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-9174fdd6-a4fa-4de8-8c49-49e9caa29af6 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:15.922723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9174fdd6-a4fa-4de8-8c49-49e9caa29af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:15.927941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c040a27bd92494db1fad3b7ae7fba45'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:15.929227 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c040a27bd92494db1fad3b7ae7fba45'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:15.930401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c040a27bd92494db1fad3b7ae7fba45'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:15.933283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 971/1938] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:15.934043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174fdd6-a4fa-4de8-8c49-49e9caa29af6 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:15.949811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c040a27bd92494db1fad3b7ae7fba45'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:15.955055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174fdd6-a4fa-4de8-8c49-49e9caa29af6 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:15.961404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e04d6a-21f8-4b92-97b4-165769f1783d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:15.962101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e04d6a-21f8-4b92-97b4-165769f1783d admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.962395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e04d6a-21f8-4b92-97b4-165769f1783d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.962657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e04d6a-21f8-4b92-97b4-165769f1783d admin admin] PATH_INFO: `/v3/projects/583d6d4f47e84344b20592e2e2001705/users/e7042e7f6a8c4b188440aa234c78456c/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.974143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 968/1939] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:15 2022] DELETE /identity/v3/users/1c040a27bd92494db1fad3b7ae7fba45 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:15.976543 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:15.982024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:15.997639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e04d6a-21f8-4b92-97b4-165769f1783d admin admin] RBAC: Authorizing `identity:create_grant(project_id=583d6d4f47e84344b20592e2e2001705, user_id=e7042e7f6a8c4b188440aa234c78456c, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.001682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e04d6a-21f8-4b92-97b4-165769f1783d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.007630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:16.039457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 972/1940] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:15 2022] PUT /identity/v3/projects/583d6d4f47e84344b20592e2e2001705/users/e7042e7f6a8c4b188440aa234c78456c/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:16.041456 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a64dd0-7741-4763-86e4-3f6526b37293 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.041776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a64dd0-7741-4763-86e4-3f6526b37293 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.042057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a64dd0-7741-4763-86e4-3f6526b37293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.070289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-37a64dd0-7741-4763-86e4-3f6526b37293 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:16.071623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-37a64dd0-7741-4763-86e4-3f6526b37293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:16.110471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a8a3b-b870-4c51-95ce-a8a65824d908 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.155670 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 973/1941] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:16.157245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196daf73-5522-4635-9233-19b7ebcbabe5 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.158051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196daf73-5522-4635-9233-19b7ebcbabe5 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.158669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196daf73-5522-4635-9233-19b7ebcbabe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.200006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-196daf73-5522-4635-9233-19b7ebcbabe5 None None] MFA Rules not processed for user `e7042e7f6a8c4b188440aa234c78456c`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:16.201478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-196daf73-5522-4635-9233-19b7ebcbabe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:16.204666 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b27a7f-f400-4431-a6e1-035aa496e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.205525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b27a7f-f400-4431-a6e1-035aa496e55b admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.205875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b27a7f-f400-4431-a6e1-035aa496e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.206150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b27a7f-f400-4431-a6e1-035aa496e55b admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.206830 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b27a7f-f400-4431-a6e1-035aa496e55b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:16.207130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b27a7f-f400-4431-a6e1-035aa496e55b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.209323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b27a7f-f400-4431-a6e1-035aa496e55b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.233447 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 969/1942] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:16.233447 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b27a7f-f400-4431-a6e1-035aa496e55b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.236051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d6b27a7f-f400-4431-a6e1-035aa496e55b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:16.265768 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:16.293160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 974/1943] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:16.294245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196daf73-5522-4635-9233-19b7ebcbabe5 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.308266 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196daf73-5522-4635-9233-19b7ebcbabe5 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.313085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c2e20-1549-478d-9f21-7ab7239fe982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.313709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c2e20-1549-478d-9f21-7ab7239fe982 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.313988 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c2e20-1549-478d-9f21-7ab7239fe982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.314260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c2e20-1549-478d-9f21-7ab7239fe982 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.314917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c2e20-1549-478d-9f21-7ab7239fe982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.316925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c2e20-1549-478d-9f21-7ab7239fe982 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.332724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b27a7f-f400-4431-a6e1-035aa496e55b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.339278 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71efb1-85ba-44f1-a3c1-6c3af44f84ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.340499 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71efb1-85ba-44f1-a3c1-6c3af44f84ba service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.340868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71efb1-85ba-44f1-a3c1-6c3af44f84ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.341359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71efb1-85ba-44f1-a3c1-6c3af44f84ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.352064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 975/1944] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:16 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:16.352064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c2e20-1549-478d-9f21-7ab7239fe982 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.354163 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e63c2e20-1549-478d-9f21-7ab7239fe982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:16.358740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71efb1-85ba-44f1-a3c1-6c3af44f84ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.361585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71efb1-85ba-44f1-a3c1-6c3af44f84ba service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.382385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 970/1945] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:16.383358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71efb1-85ba-44f1-a3c1-6c3af44f84ba service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.401246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71efb1-85ba-44f1-a3c1-6c3af44f84ba service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.441759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c2e20-1549-478d-9f21-7ab7239fe982 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.453320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ccfe5-cab0-47ed-a100-c9e333471b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.453577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ccfe5-cab0-47ed-a100-c9e333471b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.454693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ccfe5-cab0-47ed-a100-c9e333471b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.454693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ccfe5-cab0-47ed-a100-c9e333471b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.454693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ccfe5-cab0-47ed-a100-c9e333471b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.456905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ccfe5-cab0-47ed-a100-c9e333471b46 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.464250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:16.496216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 976/1946] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:16 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:16.501009 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ccfe5-cab0-47ed-a100-c9e333471b46 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.503214 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5293f29-8e20-484a-8133-45b9c2aba658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.503888 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5293f29-8e20-484a-8133-45b9c2aba658 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.504193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5293f29-8e20-484a-8133-45b9c2aba658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.504469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5293f29-8e20-484a-8133-45b9c2aba658 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.505180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5293f29-8e20-484a-8133-45b9c2aba658 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.507927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5293f29-8e20-484a-8133-45b9c2aba658 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.546812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 971/1947] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:16 2022] POST /identity/v3/users => generated 303 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:16.552015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5293f29-8e20-484a-8133-45b9c2aba658 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.565135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5293f29-8e20-484a-8133-45b9c2aba658 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.610700 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ccfe5-cab0-47ed-a100-c9e333471b46 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.616284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6752c-5733-4614-b76b-397ea6edbd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.616923 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6752c-5733-4614-b76b-397ea6edbd2a admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.617201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6752c-5733-4614-b76b-397ea6edbd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.617498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6752c-5733-4614-b76b-397ea6edbd2a admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.618178 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6752c-5733-4614-b76b-397ea6edbd2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.620308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6752c-5733-4614-b76b-397ea6edbd2a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.637225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83654041-e760-457c-aa01-e8567827fc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.638315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83654041-e760-457c-aa01-e8567827fc8a admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.639050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83654041-e760-457c-aa01-e8567827fc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.639789 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83654041-e760-457c-aa01-e8567827fc8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.641060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83654041-e760-457c-aa01-e8567827fc8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.644605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83654041-e760-457c-aa01-e8567827fc8a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.645851 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 977/1948] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:16 2022] POST /identity/v3/users => generated 294 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:16.650403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6752c-5733-4614-b76b-397ea6edbd2a admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.662729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 972/1949] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:16.662900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6752c-5733-4614-b76b-397ea6edbd2a admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.664177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83654041-e760-457c-aa01-e8567827fc8a admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.669390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db1fec-1b9f-4bd8-bbcf-3690cada2eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.670184 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db1fec-1b9f-4bd8-bbcf-3690cada2eec admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.670481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db1fec-1b9f-4bd8-bbcf-3690cada2eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.670733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db1fec-1b9f-4bd8-bbcf-3690cada2eec admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.671381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db1fec-1b9f-4bd8-bbcf-3690cada2eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.673048 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db1fec-1b9f-4bd8-bbcf-3690cada2eec admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.682951 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83654041-e760-457c-aa01-e8567827fc8a admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.685362 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 978/1950] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:16.689501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db1fec-1b9f-4bd8-bbcf-3690cada2eec admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.692664 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ea4c8-c736-4d3a-a37e-0cf10234e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.693005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ea4c8-c736-4d3a-a37e-0cf10234e82a admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.693331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ea4c8-c736-4d3a-a37e-0cf10234e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.693603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ea4c8-c736-4d3a-a37e-0cf10234e82a admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.694249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ea4c8-c736-4d3a-a37e-0cf10234e82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.695839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ea4c8-c736-4d3a-a37e-0cf10234e82a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.703743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db1fec-1b9f-4bd8-bbcf-3690cada2eec admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.708348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 973/1951] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:16.712043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19cb64-875a-49a3-97b0-9c4674521727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.712480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ea4c8-c736-4d3a-a37e-0cf10234e82a admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.713377 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19cb64-875a-49a3-97b0-9c4674521727 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.713658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19cb64-875a-49a3-97b0-9c4674521727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.713931 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19cb64-875a-49a3-97b0-9c4674521727 admin admin] PATH_INFO: `/v3/projects/0fd0e8bbce8d4f3fb29cad967f9c55a8/users/6949d404a825497faa77823e5eed12b6/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.723592 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ea4c8-c736-4d3a-a37e-0cf10234e82a admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.728991 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b70591-5f01-42e3-ad8a-efb5c77f39dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.729653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b70591-5f01-42e3-ad8a-efb5c77f39dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.729928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b70591-5f01-42e3-ad8a-efb5c77f39dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.730238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b70591-5f01-42e3-ad8a-efb5c77f39dc admin admin] PATH_INFO: `/v3/projects/b1a53ba49be8464d9bcdbfb3c74b6061/users/136d49c60f274d95ac8fbbf48e76070f/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.732134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19cb64-875a-49a3-97b0-9c4674521727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd0e8bbce8d4f3fb29cad967f9c55a8, user_id=6949d404a825497faa77823e5eed12b6, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.735117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19cb64-875a-49a3-97b0-9c4674521727 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.748069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b70591-5f01-42e3-ad8a-efb5c77f39dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a53ba49be8464d9bcdbfb3c74b6061, user_id=136d49c60f274d95ac8fbbf48e76070f, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.750866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b70591-5f01-42e3-ad8a-efb5c77f39dc admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.751847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 979/1952] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:16 2022] PUT /identity/v3/projects/0fd0e8bbce8d4f3fb29cad967f9c55a8/users/6949d404a825497faa77823e5eed12b6/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:16.757012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19cb64-875a-49a3-97b0-9c4674521727 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.766590 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 974/1953] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:16 2022] PUT /identity/v3/projects/b1a53ba49be8464d9bcdbfb3c74b6061/users/136d49c60f274d95ac8fbbf48e76070f/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:16.778853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcf9db-b0dc-4c53-b541-be441a110f27 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.779075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcf9db-b0dc-4c53-b541-be441a110f27 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.779342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcf9db-b0dc-4c53-b541-be441a110f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.809315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:16.829333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-b6dcf9db-b0dc-4c53-b541-be441a110f27 None None] MFA Rules not processed for user `136d49c60f274d95ac8fbbf48e76070f`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:16.830949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b6dcf9db-b0dc-4c53-b541-be441a110f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:16.850862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19cb64-875a-49a3-97b0-9c4674521727 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.911698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:16.927366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361aa80-ab76-4bfd-9d80-73ce88fdcc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.928834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361aa80-ab76-4bfd-9d80-73ce88fdcc5c admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.929110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361aa80-ab76-4bfd-9d80-73ce88fdcc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.929377 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361aa80-ab76-4bfd-9d80-73ce88fdcc5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.930074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361aa80-ab76-4bfd-9d80-73ce88fdcc5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.931727 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361aa80-ab76-4bfd-9d80-73ce88fdcc5c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.940149 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 975/1954] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:16 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:16.940149 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dcf9db-b0dc-4c53-b541-be441a110f27 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.940149 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b6dcf9db-b0dc-4c53-b541-be441a110f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:16.948907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 980/1955] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:16.949889 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361aa80-ab76-4bfd-9d80-73ce88fdcc5c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:16.957695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dcf9db-b0dc-4c53-b541-be441a110f27 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.960298 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361aa80-ab76-4bfd-9d80-73ce88fdcc5c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:16.963817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.964509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.964785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.964878 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd540d-a51b-4739-ba26-bdd3128c617b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:16.965056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] PATH_INFO: `/v3/projects/bae8208c34cf46ed9495a5fdbb522209` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.965578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd540d-a51b-4739-ba26-bdd3128c617b admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:16.965855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd540d-a51b-4739-ba26-bdd3128c617b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:16.966130 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd540d-a51b-4739-ba26-bdd3128c617b admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:16.966768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd540d-a51b-4739-ba26-bdd3128c617b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.966919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bae8208c34cf46ed9495a5fdbb522209)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:16.968749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd540d-a51b-4739-ba26-bdd3128c617b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:16.968880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:17.013070 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 981/1956] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:16 2022] POST /identity/v3/projects => generated 407 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:17.014917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd540d-a51b-4739-ba26-bdd3128c617b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:17.028306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bae8208c34cf46ed9495a5fdbb522209', 'request_id': 'req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:17.036395 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bae8208c34cf46ed9495a5fdbb522209', 'request_id': 'req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:17.054806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bae8208c34cf46ed9495a5fdbb522209', 'request_id': 'req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:17.068797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] The token cache is being invalidate because project bae8208c34cf46ed9495a5fdbb522209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:17.069180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:17.069866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:17.074449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 976/1957] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:16 2022] DELETE /identity/v3/projects/bae8208c34cf46ed9495a5fdbb522209 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:17.074449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:17.076209 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:17.110396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd540d-a51b-4739-ba26-bdd3128c617b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:17.115927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685d35d-be13-4585-8aad-4367c07c1328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:17.116629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685d35d-be13-4585-8aad-4367c07c1328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:17.116910 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685d35d-be13-4585-8aad-4367c07c1328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:17.117181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685d35d-be13-4585-8aad-4367c07c1328 admin admin] PATH_INFO: `/v3/projects/0fd0e8bbce8d4f3fb29cad967f9c55a8/users/6949d404a825497faa77823e5eed12b6/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:17.119285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685d35d-be13-4585-8aad-4367c07c1328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd0e8bbce8d4f3fb29cad967f9c55a8, user_id=6949d404a825497faa77823e5eed12b6, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:17.123019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685d35d-be13-4585-8aad-4367c07c1328 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:17.134084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 982/1958] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:17 2022] PUT /identity/v3/projects/0fd0e8bbce8d4f3fb29cad967f9c55a8/users/6949d404a825497faa77823e5eed12b6/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:17.135065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685d35d-be13-4585-8aad-4367c07c1328 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:17.136703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3685d35d-be13-4585-8aad-4367c07c1328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:17.152779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099fe74-ab68-4bd0-8c70-33c1b8a648cf admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:17.185284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:17.222834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e520e-c300-4f36-aca4-71d7b333b51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:17.223455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e520e-c300-4f36-aca4-71d7b333b51f admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:17.223722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e520e-c300-4f36-aca4-71d7b333b51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:17.224018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e520e-c300-4f36-aca4-71d7b333b51f admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:17.224692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e520e-c300-4f36-aca4-71d7b333b51f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:17.226747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e520e-c300-4f36-aca4-71d7b333b51f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:17.227445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685d35d-be13-4585-8aad-4367c07c1328 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:17.232358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:17.233070 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:17.233418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:17.234389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] PATH_INFO: `/v3/users/6deba2e3c3a1423ea863f0e46ce79f86` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:17.235053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6deba2e3c3a1423ea863f0e46ce79f86)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:17.237327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:17.249544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 977/1959] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:17 2022] POST /identity/v3/users => generated 310 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:17.250997 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b5fe-55a1-4b78-a9a4-9760483ba58f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:17.251285 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b5fe-55a1-4b78-a9a4-9760483ba58f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:17.251558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b5fe-55a1-4b78-a9a4-9760483ba58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:17.283847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-9cc2b5fe-55a1-4b78-a9a4-9760483ba58f None None] MFA Rules not processed for user `6949d404a825497faa77823e5eed12b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:17.285127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9cc2b5fe-55a1-4b78-a9a4-9760483ba58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:17.297869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6deba2e3c3a1423ea863f0e46ce79f86'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:17.309731 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:17.315788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6deba2e3c3a1423ea863f0e46ce79f86'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:17.330311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6deba2e3c3a1423ea863f0e46ce79f86'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:17.344945 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6deba2e3c3a1423ea863f0e46ce79f86'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:17.346068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6deba2e3c3a1423ea863f0e46ce79f86'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:17.361592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 983/1960] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:17 2022] DELETE /identity/v3/users/6deba2e3c3a1423ea863f0e46ce79f86 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:17.363074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:17.366692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:17.391175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 978/1961] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:17.404221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2b5fe-55a1-4b78-a9a4-9760483ba58f None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:17.406388 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9cc2b5fe-55a1-4b78-a9a4-9760483ba58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:17.469085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11a86e-3250-401e-8423-06c9d560be06 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:17.494531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2b5fe-55a1-4b78-a9a4-9760483ba58f None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:17.503688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da471d-a72f-4adc-b204-31ba8a34cfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:17.505185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da471d-a72f-4adc-b204-31ba8a34cfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:17.505185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da471d-a72f-4adc-b204-31ba8a34cfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:17.505417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da471d-a72f-4adc-b204-31ba8a34cfc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:17.506794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da471d-a72f-4adc-b204-31ba8a34cfc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:17.508733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da471d-a72f-4adc-b204-31ba8a34cfc1 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:17.521031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 984/1962] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:17.526578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da471d-a72f-4adc-b204-31ba8a34cfc1 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:17.554556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da471d-a72f-4adc-b204-31ba8a34cfc1 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:17.564290 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018eed54-ccd1-40e9-8ea8-f19e47c87ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:17.567123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018eed54-ccd1-40e9-8ea8-f19e47c87ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:17.567442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018eed54-ccd1-40e9-8ea8-f19e47c87ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:17.569122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018eed54-ccd1-40e9-8ea8-f19e47c87ac6 admin admin] PATH_INFO: `/v3/projects/de20cc5e2c8848b78dbb5af6ad0ab39c/users/45252f6ff97c4a669159929459849647/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:17.602499 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef907f2-3406-4137-8c01-ddb917945305 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:17.603194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef907f2-3406-4137-8c01-ddb917945305 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:17.603578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef907f2-3406-4137-8c01-ddb917945305 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:17.603855 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef907f2-3406-4137-8c01-ddb917945305 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:17.605219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018eed54-ccd1-40e9-8ea8-f19e47c87ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de20cc5e2c8848b78dbb5af6ad0ab39c, user_id=45252f6ff97c4a669159929459849647, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:17.608725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018eed54-ccd1-40e9-8ea8-f19e47c87ac6 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:17.611518 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3ef907f2-3406-4137-8c01-ddb917945305 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:17.643506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 985/1963] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:17 2022] PUT /identity/v3/projects/de20cc5e2c8848b78dbb5af6ad0ab39c/users/45252f6ff97c4a669159929459849647/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:17.645567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018eed54-ccd1-40e9-8ea8-f19e47c87ac6 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:17.650121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:17.760472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef907f2-3406-4137-8c01-ddb917945305 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:17.761053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef907f2-3406-4137-8c01-ddb917945305 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:17.787950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 979/1964] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:17.800940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018eed54-ccd1-40e9-8ea8-f19e47c87ac6 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:17.808044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117c151-4211-4729-b3d2-a17ee977df31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:17.808696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117c151-4211-4729-b3d2-a17ee977df31 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:17.808976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117c151-4211-4729-b3d2-a17ee977df31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:17.809422 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117c151-4211-4729-b3d2-a17ee977df31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:17.809902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117c151-4211-4729-b3d2-a17ee977df31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:17.811531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117c151-4211-4729-b3d2-a17ee977df31 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:17.829486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 986/1965] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:17.829723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef907f2-3406-4137-8c01-ddb917945305 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:17.854358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef907f2-3406-4137-8c01-ddb917945305 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:17.866962 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bdf11-3f01-4266-91d7-b4b7fc55587c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:17.867762 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bdf11-3f01-4266-91d7-b4b7fc55587c admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:17.868075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bdf11-3f01-4266-91d7-b4b7fc55587c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:17.868356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bdf11-3f01-4266-91d7-b4b7fc55587c admin admin] PATH_INFO: `/v3/projects/de20cc5e2c8848b78dbb5af6ad0ab39c/users/45252f6ff97c4a669159929459849647/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:17.873580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bdf11-3f01-4266-91d7-b4b7fc55587c admin admin] RBAC: Authorizing `identity:create_grant(project_id=de20cc5e2c8848b78dbb5af6ad0ab39c, user_id=45252f6ff97c4a669159929459849647, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:17.877461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bdf11-3f01-4266-91d7-b4b7fc55587c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:17.894598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 980/1966] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:17 2022] PUT /identity/v3/projects/de20cc5e2c8848b78dbb5af6ad0ab39c/users/45252f6ff97c4a669159929459849647/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:17.906340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97202ab0-0b7b-4f8d-b4b1-fef3ef14ebc2 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:17.907255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97202ab0-0b7b-4f8d-b4b1-fef3ef14ebc2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:17.907778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97202ab0-0b7b-4f8d-b4b1-fef3ef14ebc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:17.966469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-97202ab0-0b7b-4f8d-b4b1-fef3ef14ebc2 None None] MFA Rules not processed for user `45252f6ff97c4a669159929459849647`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:17.968438 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-97202ab0-0b7b-4f8d-b4b1-fef3ef14ebc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:18.024217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bdf11-3f01-4266-91d7-b4b7fc55587c admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:18.026204 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-298bdf11-3f01-4266-91d7-b4b7fc55587c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:18.035247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:18.115776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 987/1967] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:17 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:18.149280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bdf11-3f01-4266-91d7-b4b7fc55587c admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:18.155707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef870421-a2d8-4066-98bd-b9d80094d73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:18.156392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef870421-a2d8-4066-98bd-b9d80094d73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:18.156670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef870421-a2d8-4066-98bd-b9d80094d73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:18.157323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef870421-a2d8-4066-98bd-b9d80094d73b admin admin] PATH_INFO: `/v3/projects/cd8b69f0275843da9386e6be64bce685` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:18.158910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef870421-a2d8-4066-98bd-b9d80094d73b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8b69f0275843da9386e6be64bce685)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:18.161089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef870421-a2d8-4066-98bd-b9d80094d73b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:18.218284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97202ab0-0b7b-4f8d-b4b1-fef3ef14ebc2 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:18.219913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-97202ab0-0b7b-4f8d-b4b1-fef3ef14ebc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:18.225851 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ef870421-a2d8-4066-98bd-b9d80094d73b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8b69f0275843da9386e6be64bce685', 'request_id': 'req-ef870421-a2d8-4066-98bd-b9d80094d73b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:18.232285 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ef870421-a2d8-4066-98bd-b9d80094d73b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8b69f0275843da9386e6be64bce685', 'request_id': 'req-ef870421-a2d8-4066-98bd-b9d80094d73b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:18.238582 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ef870421-a2d8-4066-98bd-b9d80094d73b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8b69f0275843da9386e6be64bce685', 'request_id': 'req-ef870421-a2d8-4066-98bd-b9d80094d73b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:18.255036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ef870421-a2d8-4066-98bd-b9d80094d73b admin admin] The token cache is being invalidate because project cd8b69f0275843da9386e6be64bce685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:18.255036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ef870421-a2d8-4066-98bd-b9d80094d73b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:18.256642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ef870421-a2d8-4066-98bd-b9d80094d73b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:18.259225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 981/1968] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:18 2022] DELETE /identity/v3/projects/cd8b69f0275843da9386e6be64bce685 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:18.264830 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b1233-b28c-40dc-8794-ceb9b588134d None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:18.265237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b1233-b28c-40dc-8794-ceb9b588134d None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:18.265568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b1233-b28c-40dc-8794-ceb9b588134d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:18.297702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c34b1233-b28c-40dc-8794-ceb9b588134d None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:18.299321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c34b1233-b28c-40dc-8794-ceb9b588134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:18.319790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97202ab0-0b7b-4f8d-b4b1-fef3ef14ebc2 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:18.367167 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:18.448391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652e150-5bb1-4bb6-a838-8abe3037a504 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:18.448657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 982/1969] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:18.448747 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652e150-5bb1-4bb6-a838-8abe3037a504 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:18.448747 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652e150-5bb1-4bb6-a838-8abe3037a504 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:18.448747 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652e150-5bb1-4bb6-a838-8abe3037a504 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:18.448747 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f652e150-5bb1-4bb6-a838-8abe3037a504 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:18.451366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014b7e1-41cd-4b25-af9b-b77fc4e46c8a None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:18.451853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014b7e1-41cd-4b25-af9b-b77fc4e46c8a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:18.452150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014b7e1-41cd-4b25-af9b-b77fc4e46c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:18.483759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c014b7e1-41cd-4b25-af9b-b77fc4e46c8a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:18.488019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c014b7e1-41cd-4b25-af9b-b77fc4e46c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:18.501026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 983/1970] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:18.506541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014b7e1-41cd-4b25-af9b-b77fc4e46c8a None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:18.520089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014b7e1-41cd-4b25-af9b-b77fc4e46c8a None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:18.529957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43448ebf-ad49-4962-92d6-fd43aa3e8d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:18.531145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43448ebf-ad49-4962-92d6-fd43aa3e8d82 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:18.531849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43448ebf-ad49-4962-92d6-fd43aa3e8d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:18.532334 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43448ebf-ad49-4962-92d6-fd43aa3e8d82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:18.533638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43448ebf-ad49-4962-92d6-fd43aa3e8d82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:18.534225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43448ebf-ad49-4962-92d6-fd43aa3e8d82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:18.536071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43448ebf-ad49-4962-92d6-fd43aa3e8d82 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:18.553743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652e150-5bb1-4bb6-a838-8abe3037a504 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:18.555518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652e150-5bb1-4bb6-a838-8abe3037a504 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:18.561124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 984/1971] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:18.566139 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43448ebf-ad49-4962-92d6-fd43aa3e8d82 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:18.582493 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43448ebf-ad49-4962-92d6-fd43aa3e8d82 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:18.582715 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 988/1972] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:18 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:18.590636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5ef86-260c-467a-ad29-baa026d621c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:18.592362 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5ef86-260c-467a-ad29-baa026d621c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:18.592904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5ef86-260c-467a-ad29-baa026d621c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:18.593428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5ef86-260c-467a-ad29-baa026d621c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:18.594538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5ef86-260c-467a-ad29-baa026d621c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:18.597162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5ef86-260c-467a-ad29-baa026d621c5 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:18.655477 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 985/1973] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:18 2022] POST /identity/v3/projects => generated 361 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:18.659334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652e150-5bb1-4bb6-a838-8abe3037a504 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:18.671983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652e150-5bb1-4bb6-a838-8abe3037a504 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:18.752843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:18.787251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5d735-e8d2-4ccc-8c4d-fece45021425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:18.788244 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5d735-e8d2-4ccc-8c4d-fece45021425 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:18.788698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5d735-e8d2-4ccc-8c4d-fece45021425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:18.789000 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5d735-e8d2-4ccc-8c4d-fece45021425 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:18.789685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5d735-e8d2-4ccc-8c4d-fece45021425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:18.791723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5d735-e8d2-4ccc-8c4d-fece45021425 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:18.818860 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 989/1974] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:18 2022] POST /identity/v3/users => generated 281 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:18.823766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5ef86-260c-467a-ad29-baa026d621c5 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:18.841750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5ef86-260c-467a-ad29-baa026d621c5 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:18.848583 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae113e-975e-4579-bcd8-5f31f0e4fb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:18.849815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae113e-975e-4579-bcd8-5f31f0e4fb14 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:18.850825 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae113e-975e-4579-bcd8-5f31f0e4fb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:18.851170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae113e-975e-4579-bcd8-5f31f0e4fb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:18.853383 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae113e-975e-4579-bcd8-5f31f0e4fb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:18.853681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae113e-975e-4579-bcd8-5f31f0e4fb14 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:18.864815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 986/1975] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:18.875791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5d735-e8d2-4ccc-8c4d-fece45021425 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:18.887531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5d735-e8d2-4ccc-8c4d-fece45021425 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:18.892875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae113e-975e-4579-bcd8-5f31f0e4fb14 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:18.894610 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6872d3c-969d-477f-aa87-97fcb4513edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:18.895346 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6872d3c-969d-477f-aa87-97fcb4513edf admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:18.895456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b8ae113e-975e-4579-bcd8-5f31f0e4fb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:18.895799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6872d3c-969d-477f-aa87-97fcb4513edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:18.896758 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6872d3c-969d-477f-aa87-97fcb4513edf admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:18.897514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6872d3c-969d-477f-aa87-97fcb4513edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:18.899229 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6872d3c-969d-477f-aa87-97fcb4513edf admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:18.913922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 990/1976] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:18.914911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6872d3c-969d-477f-aa87-97fcb4513edf admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:18.928222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6872d3c-969d-477f-aa87-97fcb4513edf admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:18.937157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f04c3b-d159-47b8-be03-da63acf79547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:18.938556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f04c3b-d159-47b8-be03-da63acf79547 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:18.939132 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f04c3b-d159-47b8-be03-da63acf79547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:18.939683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f04c3b-d159-47b8-be03-da63acf79547 admin admin] PATH_INFO: `/v3/projects/ce7588fefe9b40dca81049f84b9bf1e9/users/dcb69b1bc11b478a936d12786b3aa8c6/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:18.965364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f04c3b-d159-47b8-be03-da63acf79547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7588fefe9b40dca81049f84b9bf1e9, user_id=dcb69b1bc11b478a936d12786b3aa8c6, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:18.968415 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f04c3b-d159-47b8-be03-da63acf79547 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:18.977062 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae113e-975e-4579-bcd8-5f31f0e4fb14 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:18.989009 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 991/1977] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:18 2022] PUT /identity/v3/projects/ce7588fefe9b40dca81049f84b9bf1e9/users/dcb69b1bc11b478a936d12786b3aa8c6/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:19.006557 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5590e1-d588-4c5c-9f37-70a992747e58 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:19.006928 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5590e1-d588-4c5c-9f37-70a992747e58 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:19.007321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5590e1-d588-4c5c-9f37-70a992747e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:19.049567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ec5590e1-d588-4c5c-9f37-70a992747e58 None None] MFA Rules not processed for user `dcb69b1bc11b478a936d12786b3aa8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:19.050027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ec5590e1-d588-4c5c-9f37-70a992747e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:19.061471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de83ed4-cc5c-41b6-bacc-0d7680d4fa17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:19.062501 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de83ed4-cc5c-41b6-bacc-0d7680d4fa17 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:19.062860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de83ed4-cc5c-41b6-bacc-0d7680d4fa17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:19.063275 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de83ed4-cc5c-41b6-bacc-0d7680d4fa17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:19.065635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9de83ed4-cc5c-41b6-bacc-0d7680d4fa17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:19.186429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:19.212716 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de83ed4-cc5c-41b6-bacc-0d7680d4fa17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:19.220856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 992/1978] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:19.221063 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de83ed4-cc5c-41b6-bacc-0d7680d4fa17 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:19.229676 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 987/1979] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:18 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:19.541384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5590e1-d588-4c5c-9f37-70a992747e58 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:19.542874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ec5590e1-d588-4c5c-9f37-70a992747e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:19.569816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5590e1-d588-4c5c-9f37-70a992747e58 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:19.582778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746f6fe-058f-4422-8927-eff6db61f9e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:19.584201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746f6fe-058f-4422-8927-eff6db61f9e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:19.585295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746f6fe-058f-4422-8927-eff6db61f9e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:19.585718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746f6fe-058f-4422-8927-eff6db61f9e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:19.596540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746f6fe-058f-4422-8927-eff6db61f9e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:19.598436 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746f6fe-058f-4422-8927-eff6db61f9e5 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:19.609858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 993/1980] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:19.640160 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de83ed4-cc5c-41b6-bacc-0d7680d4fa17 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:19.768222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de83ed4-cc5c-41b6-bacc-0d7680d4fa17 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:19.774346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:19.775077 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:19.775394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:19.775619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] PATH_INFO: `/v3/users/dcb69b1bc11b478a936d12786b3aa8c6` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:19.776873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcb69b1bc11b478a936d12786b3aa8c6)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:19.779524 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:19.865936 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcb69b1bc11b478a936d12786b3aa8c6'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:19.879301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcb69b1bc11b478a936d12786b3aa8c6'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:19.880069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcb69b1bc11b478a936d12786b3aa8c6'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:19.881404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcb69b1bc11b478a936d12786b3aa8c6'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:19.915815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcb69b1bc11b478a936d12786b3aa8c6'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:19.953301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 988/1981] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:19 2022] DELETE /identity/v3/users/dcb69b1bc11b478a936d12786b3aa8c6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:19.959628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5c32-d176-48b8-8c3a-fd1ad8c3ca86 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:19.963161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5c32-d176-48b8-8c3a-fd1ad8c3ca86 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:19.964084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5c32-d176-48b8-8c3a-fd1ad8c3ca86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:20.017520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d1ce5c32-d176-48b8-8c3a-fd1ad8c3ca86 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:20.018857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d1ce5c32-d176-48b8-8c3a-fd1ad8c3ca86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:20.160169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 994/1982] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:20.178476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:20.182492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:20.287359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac839f-1901-4592-a8d2-79f971b6540e admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:20.292193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aa2cc-87ac-4054-8201-2e4161faea8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:20.292875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa2cc-87ac-4054-8201-2e4161faea8a service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:20.293386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa2cc-87ac-4054-8201-2e4161faea8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:20.293733 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa2cc-87ac-4054-8201-2e4161faea8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:20.305629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941aa2cc-87ac-4054-8201-2e4161faea8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:20.307352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941aa2cc-87ac-4054-8201-2e4161faea8a service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:20.319302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 989/1983] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:20.746874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5c32-d176-48b8-8c3a-fd1ad8c3ca86 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:20.749220 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d1ce5c32-d176-48b8-8c3a-fd1ad8c3ca86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:20.770747 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5c32-d176-48b8-8c3a-fd1ad8c3ca86 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:20.780985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309ee58-e07a-47ae-a7fd-47fe3af052ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:20.781944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309ee58-e07a-47ae-a7fd-47fe3af052ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:20.783488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309ee58-e07a-47ae-a7fd-47fe3af052ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:20.783488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309ee58-e07a-47ae-a7fd-47fe3af052ae admin admin] PATH_INFO: `/v3/projects/ce7588fefe9b40dca81049f84b9bf1e9` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:20.786471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309ee58-e07a-47ae-a7fd-47fe3af052ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce7588fefe9b40dca81049f84b9bf1e9)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:20.789483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309ee58-e07a-47ae-a7fd-47fe3af052ae admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:20.791029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:20.932646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c309ee58-e07a-47ae-a7fd-47fe3af052ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce7588fefe9b40dca81049f84b9bf1e9', 'request_id': 'req-c309ee58-e07a-47ae-a7fd-47fe3af052ae'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:20.948378 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c309ee58-e07a-47ae-a7fd-47fe3af052ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce7588fefe9b40dca81049f84b9bf1e9', 'request_id': 'req-c309ee58-e07a-47ae-a7fd-47fe3af052ae'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:20.957845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c309ee58-e07a-47ae-a7fd-47fe3af052ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce7588fefe9b40dca81049f84b9bf1e9', 'request_id': 'req-c309ee58-e07a-47ae-a7fd-47fe3af052ae'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:21.000773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c309ee58-e07a-47ae-a7fd-47fe3af052ae admin admin] The token cache is being invalidate because project ce7588fefe9b40dca81049f84b9bf1e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:21.001084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c309ee58-e07a-47ae-a7fd-47fe3af052ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:21.002001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c309ee58-e07a-47ae-a7fd-47fe3af052ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:21.006736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 995/1984] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:20 2022] DELETE /identity/v3/projects/ce7588fefe9b40dca81049f84b9bf1e9 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:21.012167 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f4311-7706-4b6f-be32-17b1218850a6 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:21.012877 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f4311-7706-4b6f-be32-17b1218850a6 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:21.012877 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f4311-7706-4b6f-be32-17b1218850a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:21.051368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-ea1f4311-7706-4b6f-be32-17b1218850a6 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:21.054356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ea1f4311-7706-4b6f-be32-17b1218850a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:21.065603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:21.170974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 990/1985] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:21.185453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede704c-0622-44bf-a905-6053712e85e2 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:21.185769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede704c-0622-44bf-a905-6053712e85e2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:21.186062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede704c-0622-44bf-a905-6053712e85e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:21.210463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-cede704c-0622-44bf-a905-6053712e85e2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:21.211471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-cede704c-0622-44bf-a905-6053712e85e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:21.222457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 996/1986] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:21.228614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f4311-7706-4b6f-be32-17b1218850a6 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:21.251499 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f4311-7706-4b6f-be32-17b1218850a6 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:21.257994 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3975f-e888-4dc0-8891-ad07d7961f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:21.258973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3975f-e888-4dc0-8891-ad07d7961f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:21.259406 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3975f-e888-4dc0-8891-ad07d7961f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:21.259849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3975f-e888-4dc0-8891-ad07d7961f0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:21.261291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3975f-e888-4dc0-8891-ad07d7961f0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:21.261786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3975f-e888-4dc0-8891-ad07d7961f0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:21.264495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3975f-e888-4dc0-8891-ad07d7961f0f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:21.280259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 991/1987] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:21.287598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede704c-0622-44bf-a905-6053712e85e2 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:21.304999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede704c-0622-44bf-a905-6053712e85e2 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:21.312444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594daf9-c18a-4190-9685-e62b08387492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:21.313425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594daf9-c18a-4190-9685-e62b08387492 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:21.313883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594daf9-c18a-4190-9685-e62b08387492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:21.314335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594daf9-c18a-4190-9685-e62b08387492 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:21.315442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594daf9-c18a-4190-9685-e62b08387492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:21.319158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594daf9-c18a-4190-9685-e62b08387492 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:21.371313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 997/1988] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:21 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:21.372705 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3975f-e888-4dc0-8891-ad07d7961f0f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:21.385171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3975f-e888-4dc0-8891-ad07d7961f0f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:21.469966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f488ef4-a7c5-4866-83d3-e8469a9b1b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:21.471052 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f488ef4-a7c5-4866-83d3-e8469a9b1b74 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:21.471544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f488ef4-a7c5-4866-83d3-e8469a9b1b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:21.472068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f488ef4-a7c5-4866-83d3-e8469a9b1b74 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:21.473233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f488ef4-a7c5-4866-83d3-e8469a9b1b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:21.477163 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f488ef4-a7c5-4866-83d3-e8469a9b1b74 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:21.514866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 992/1989] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:21 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:21.519097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594daf9-c18a-4190-9685-e62b08387492 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:21.533341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594daf9-c18a-4190-9685-e62b08387492 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:21.540122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a9533-ca1d-4dd9-897f-acd0c8dcd452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:21.540847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a9533-ca1d-4dd9-897f-acd0c8dcd452 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:21.541121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a9533-ca1d-4dd9-897f-acd0c8dcd452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:21.541386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a9533-ca1d-4dd9-897f-acd0c8dcd452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:21.542416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a9533-ca1d-4dd9-897f-acd0c8dcd452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:21.543660 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a9533-ca1d-4dd9-897f-acd0c8dcd452 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:21.555942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 998/1990] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:21.561650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f488ef4-a7c5-4866-83d3-e8469a9b1b74 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:21.575000 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f488ef4-a7c5-4866-83d3-e8469a9b1b74 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:21.584326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474e3e8-9e06-4f53-ae73-6f3b41761781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:21.584326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474e3e8-9e06-4f53-ae73-6f3b41761781 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:21.584326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474e3e8-9e06-4f53-ae73-6f3b41761781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:21.584326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474e3e8-9e06-4f53-ae73-6f3b41761781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:21.584985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474e3e8-9e06-4f53-ae73-6f3b41761781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:21.585912 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474e3e8-9e06-4f53-ae73-6f3b41761781 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:21.598071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 993/1991] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:21.602541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a9533-ca1d-4dd9-897f-acd0c8dcd452 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:21.612985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a9533-ca1d-4dd9-897f-acd0c8dcd452 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:21.619275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736601d9-acc0-4d61-8f5e-afa226b91b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:21.619958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736601d9-acc0-4d61-8f5e-afa226b91b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:21.620276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736601d9-acc0-4d61-8f5e-afa226b91b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:21.620546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736601d9-acc0-4d61-8f5e-afa226b91b51 admin admin] PATH_INFO: `/v3/projects/e01fa2a12bff48b8b920872e4c6ae588/users/4aaeef813f704993a4d8951e9013d7dd/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:21.641778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736601d9-acc0-4d61-8f5e-afa226b91b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01fa2a12bff48b8b920872e4c6ae588, user_id=4aaeef813f704993a4d8951e9013d7dd, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:21.645333 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736601d9-acc0-4d61-8f5e-afa226b91b51 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:21.663476 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 999/1992] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:21 2022] PUT /identity/v3/projects/e01fa2a12bff48b8b920872e4c6ae588/users/4aaeef813f704993a4d8951e9013d7dd/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:21.674559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf7667-9066-4b97-b425-b61a038aec28 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:21.675011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf7667-9066-4b97-b425-b61a038aec28 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:21.675410 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf7667-9066-4b97-b425-b61a038aec28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:21.710001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8baf7667-9066-4b97-b425-b61a038aec28 None None] MFA Rules not processed for user `4aaeef813f704993a4d8951e9013d7dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:21.711344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8baf7667-9066-4b97-b425-b61a038aec28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:21.796460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 994/1993] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:21.801120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736601d9-acc0-4d61-8f5e-afa226b91b51 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:21.891611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736601d9-acc0-4d61-8f5e-afa226b91b51 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:21.897350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15ba10-c46d-4c9f-be4a-72b2ebdb5de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:21.897984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15ba10-c46d-4c9f-be4a-72b2ebdb5de9 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:21.898270 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15ba10-c46d-4c9f-be4a-72b2ebdb5de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:21.898546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15ba10-c46d-4c9f-be4a-72b2ebdb5de9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:21.899198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15ba10-c46d-4c9f-be4a-72b2ebdb5de9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:21.901338 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15ba10-c46d-4c9f-be4a-72b2ebdb5de9 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:21.937370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1000/1994] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:21 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:21.939961 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf7667-9066-4b97-b425-b61a038aec28 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:21.955870 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf7667-9066-4b97-b425-b61a038aec28 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:21.963638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:22.043818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8328da-2eb2-4a42-b8de-12a466742349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:22.044535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8328da-2eb2-4a42-b8de-12a466742349 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:22.044751 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8328da-2eb2-4a42-b8de-12a466742349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:22.045036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8328da-2eb2-4a42-b8de-12a466742349 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:22.045684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8328da-2eb2-4a42-b8de-12a466742349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:22.047609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8328da-2eb2-4a42-b8de-12a466742349 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:22.072351 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 995/1995] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:21 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:22.076325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15ba10-c46d-4c9f-be4a-72b2ebdb5de9 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:22.088423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15ba10-c46d-4c9f-be4a-72b2ebdb5de9 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:22.097558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e5c5a-58b4-4a56-a1c5-35305e5d1939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:22.098407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e5c5a-58b4-4a56-a1c5-35305e5d1939 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:22.098713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e5c5a-58b4-4a56-a1c5-35305e5d1939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:22.099612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e5c5a-58b4-4a56-a1c5-35305e5d1939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:22.100564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e5c5a-58b4-4a56-a1c5-35305e5d1939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:22.102213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e5c5a-58b4-4a56-a1c5-35305e5d1939 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:22.110368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:22.118692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1001/1996] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:22.123989 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8328da-2eb2-4a42-b8de-12a466742349 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:22.135930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8328da-2eb2-4a42-b8de-12a466742349 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:22.142370 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25406960-a20b-4c2f-9fe9-4f4664eda075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:22.143069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25406960-a20b-4c2f-9fe9-4f4664eda075 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:22.143355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25406960-a20b-4c2f-9fe9-4f4664eda075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:22.143628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25406960-a20b-4c2f-9fe9-4f4664eda075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:22.144670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25406960-a20b-4c2f-9fe9-4f4664eda075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:22.147234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25406960-a20b-4c2f-9fe9-4f4664eda075 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:22.159430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 996/1997] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:22.164641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e5c5a-58b4-4a56-a1c5-35305e5d1939 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:22.182028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e5c5a-58b4-4a56-a1c5-35305e5d1939 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:22.187816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cd4f2-81ab-42cf-beed-44541cd0d3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:22.188535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cd4f2-81ab-42cf-beed-44541cd0d3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:22.188817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cd4f2-81ab-42cf-beed-44541cd0d3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:22.189089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cd4f2-81ab-42cf-beed-44541cd0d3b8 admin admin] PATH_INFO: `/v3/projects/cb88f173834a4513983bc7aef169dcd4/users/02b64cf20db64fc39e3ed67f6430dc8d/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:22.205803 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960cd4f2-81ab-42cf-beed-44541cd0d3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb88f173834a4513983bc7aef169dcd4, user_id=02b64cf20db64fc39e3ed67f6430dc8d, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:22.209108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960cd4f2-81ab-42cf-beed-44541cd0d3b8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:22.224840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1002/1998] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:22 2022] PUT /identity/v3/projects/cb88f173834a4513983bc7aef169dcd4/users/02b64cf20db64fc39e3ed67f6430dc8d/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:22.242725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d68bb-2673-4e12-9aa6-abbbf1380654 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:22.243006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d68bb-2673-4e12-9aa6-abbbf1380654 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:22.243280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d68bb-2673-4e12-9aa6-abbbf1380654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:22.276361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c19d68bb-2673-4e12-9aa6-abbbf1380654 None None] MFA Rules not processed for user `02b64cf20db64fc39e3ed67f6430dc8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:22.277818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c19d68bb-2673-4e12-9aa6-abbbf1380654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:22.351523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 997/1999] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:22.355908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cd4f2-81ab-42cf-beed-44541cd0d3b8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:22.431539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cd4f2-81ab-42cf-beed-44541cd0d3b8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:22.437510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c3927-3e32-4e56-afb2-e55faf2757f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:22.438414 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c3927-3e32-4e56-afb2-e55faf2757f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:22.438695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c3927-3e32-4e56-afb2-e55faf2757f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:22.438961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c3927-3e32-4e56-afb2-e55faf2757f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:22.439621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c3927-3e32-4e56-afb2-e55faf2757f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:22.441569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c3927-3e32-4e56-afb2-e55faf2757f1 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:22.483228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1003/2000] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:22 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:22.488067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d68bb-2673-4e12-9aa6-abbbf1380654 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:22.502444 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d68bb-2673-4e12-9aa6-abbbf1380654 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:22.568910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecd6dc-ffa7-4ba6-8be1-474377c03bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:22.569734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecd6dc-ffa7-4ba6-8be1-474377c03bac admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:22.570012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecd6dc-ffa7-4ba6-8be1-474377c03bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:22.570570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecd6dc-ffa7-4ba6-8be1-474377c03bac admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:22.571231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecd6dc-ffa7-4ba6-8be1-474377c03bac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:22.573216 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecd6dc-ffa7-4ba6-8be1-474377c03bac admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:22.612086 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 998/2001] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:22 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:22.619927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c3927-3e32-4e56-afb2-e55faf2757f1 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:22.644235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c3927-3e32-4e56-afb2-e55faf2757f1 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:22.653632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57719e30-925c-46ca-bded-536a0692122a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:22.654918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57719e30-925c-46ca-bded-536a0692122a admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:22.655427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57719e30-925c-46ca-bded-536a0692122a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:22.656120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57719e30-925c-46ca-bded-536a0692122a admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:22.657325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57719e30-925c-46ca-bded-536a0692122a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:22.665756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57719e30-925c-46ca-bded-536a0692122a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:22.690577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1004/2002] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:22.694781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecd6dc-ffa7-4ba6-8be1-474377c03bac admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:22.715277 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecd6dc-ffa7-4ba6-8be1-474377c03bac admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:22.726931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3482684-5491-4292-8ee6-e8248d2f573f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:22.727746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3482684-5491-4292-8ee6-e8248d2f573f admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:22.728134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3482684-5491-4292-8ee6-e8248d2f573f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:22.728469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3482684-5491-4292-8ee6-e8248d2f573f admin admin] PATH_INFO: `/v3/projects/19a0ccb72ee0460695346bdd507c5b2c/users/ff3d8c44932847e1ac8ab6d5dbf5c2cb/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:22.731502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:22.750338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3482684-5491-4292-8ee6-e8248d2f573f admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a0ccb72ee0460695346bdd507c5b2c, user_id=ff3d8c44932847e1ac8ab6d5dbf5c2cb, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:22.753343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3482684-5491-4292-8ee6-e8248d2f573f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:22.776424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 999/2003] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:22 2022] PUT /identity/v3/projects/19a0ccb72ee0460695346bdd507c5b2c/users/ff3d8c44932847e1ac8ab6d5dbf5c2cb/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:22.782669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57719e30-925c-46ca-bded-536a0692122a admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:22.800612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57719e30-925c-46ca-bded-536a0692122a admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:22.898120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ef635-2e7a-44b8-8b9c-86e9af853330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:22.899095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ef635-2e7a-44b8-8b9c-86e9af853330 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:22.899383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ef635-2e7a-44b8-8b9c-86e9af853330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:22.901258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ef635-2e7a-44b8-8b9c-86e9af853330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:22.902051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ef635-2e7a-44b8-8b9c-86e9af853330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:22.903949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ef635-2e7a-44b8-8b9c-86e9af853330 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:22.919216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1005/2004] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:22.924723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3482684-5491-4292-8ee6-e8248d2f573f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:22.936910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3482684-5491-4292-8ee6-e8248d2f573f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:22.942147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f3e67-c4f9-4a83-aee9-4c27663eb09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:22.943487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f3e67-c4f9-4a83-aee9-4c27663eb09a admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:22.943988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f3e67-c4f9-4a83-aee9-4c27663eb09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:22.944458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f3e67-c4f9-4a83-aee9-4c27663eb09a admin admin] PATH_INFO: `/v3/projects/19a0ccb72ee0460695346bdd507c5b2c/users/ff3d8c44932847e1ac8ab6d5dbf5c2cb/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:22.946505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f3e67-c4f9-4a83-aee9-4c27663eb09a admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a0ccb72ee0460695346bdd507c5b2c, user_id=ff3d8c44932847e1ac8ab6d5dbf5c2cb, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:22.949207 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f3e67-c4f9-4a83-aee9-4c27663eb09a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:22.966086 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1000/2005] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:22 2022] PUT /identity/v3/projects/19a0ccb72ee0460695346bdd507c5b2c/users/ff3d8c44932847e1ac8ab6d5dbf5c2cb/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:22.976460 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275ca6a-973a-402d-a8cf-135e24bbe5ab None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:22.976742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275ca6a-973a-402d-a8cf-135e24bbe5ab None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:22.977021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275ca6a-973a-402d-a8cf-135e24bbe5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:23.010556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-4275ca6a-973a-402d-a8cf-135e24bbe5ab None None] MFA Rules not processed for user `ff3d8c44932847e1ac8ab6d5dbf5c2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:23.011117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4275ca6a-973a-402d-a8cf-135e24bbe5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:23.073708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:23.125099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1006/2006] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:23.138476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f3e67-c4f9-4a83-aee9-4c27663eb09a admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:23.141166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d64f3e67-c4f9-4a83-aee9-4c27663eb09a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:23.232317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f3e67-c4f9-4a83-aee9-4c27663eb09a admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:23.236712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa802f6-01ca-46b9-8f0f-883d56184e7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:23.237502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa802f6-01ca-46b9-8f0f-883d56184e7b service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:23.237777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa802f6-01ca-46b9-8f0f-883d56184e7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:23.238042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa802f6-01ca-46b9-8f0f-883d56184e7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:23.308766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa802f6-01ca-46b9-8f0f-883d56184e7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:23.310779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa802f6-01ca-46b9-8f0f-883d56184e7b service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:23.323462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1001/2007] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:25.266844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275ca6a-973a-402d-a8cf-135e24bbe5ab None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:25.277719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275ca6a-973a-402d-a8cf-135e24bbe5ab None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:25.281846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b954e-4e58-4d51-8111-1c9638bc7f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:25.282477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b954e-4e58-4d51-8111-1c9638bc7f9c service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:25.282759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b954e-4e58-4d51-8111-1c9638bc7f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:25.283032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b954e-4e58-4d51-8111-1c9638bc7f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:25.284909 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2c4b954e-4e58-4d51-8111-1c9638bc7f9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:25.384783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b954e-4e58-4d51-8111-1c9638bc7f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:25.386397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b954e-4e58-4d51-8111-1c9638bc7f9c service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:25.425161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1007/2008] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:25.506479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa802f6-01ca-46b9-8f0f-883d56184e7b service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:25.508176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4aa802f6-01ca-46b9-8f0f-883d56184e7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:25.520609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa802f6-01ca-46b9-8f0f-883d56184e7b service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:25.524973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025d266-d79f-471f-9943-9a4cb1b70cc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:25.525667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025d266-d79f-471f-9943-9a4cb1b70cc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:25.526005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025d266-d79f-471f-9943-9a4cb1b70cc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:25.526435 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025d266-d79f-471f-9943-9a4cb1b70cc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:25.528660 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2025d266-d79f-471f-9943-9a4cb1b70cc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:25.622222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025d266-d79f-471f-9943-9a4cb1b70cc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:25.624874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025d266-d79f-471f-9943-9a4cb1b70cc4 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:25.647696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1002/2009] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:32.674379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b954e-4e58-4d51-8111-1c9638bc7f9c service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:32.683557 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b954e-4e58-4d51-8111-1c9638bc7f9c service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:32.688378 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8090ce-d71d-40ce-a428-21f2e683915c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:32.689083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8090ce-d71d-40ce-a428-21f2e683915c service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:32.689438 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8090ce-d71d-40ce-a428-21f2e683915c service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:32.689630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8090ce-d71d-40ce-a428-21f2e683915c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:32.809544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8090ce-d71d-40ce-a428-21f2e683915c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:32.811027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8090ce-d71d-40ce-a428-21f2e683915c service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:32.831199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1008/2010] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:35.590054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025d266-d79f-471f-9943-9a4cb1b70cc4 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:35.590514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2025d266-d79f-471f-9943-9a4cb1b70cc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:35.611136 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:35.662267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8090ce-d71d-40ce-a428-21f2e683915c service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:35.662843 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025d266-d79f-471f-9943-9a4cb1b70cc4 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:35.668514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:35.669151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:35.669428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:35.669696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] PATH_INFO: `/v3/users/136d49c60f274d95ac8fbbf48e76070f` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:35.670829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=136d49c60f274d95ac8fbbf48e76070f)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:35.671468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8090ce-d71d-40ce-a428-21f2e683915c service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:35.672778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:35.677654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46aba6-3bb4-4e08-b519-538940283862 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:35.678373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46aba6-3bb4-4e08-b519-538940283862 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:35.678645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46aba6-3bb4-4e08-b519-538940283862 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:35.678914 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46aba6-3bb4-4e08-b519-538940283862 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:35.696351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46aba6-3bb4-4e08-b519-538940283862 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:35.698108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46aba6-3bb4-4e08-b519-538940283862 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:35.710984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1009/2011] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:35.734051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '136d49c60f274d95ac8fbbf48e76070f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:35.741994 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '136d49c60f274d95ac8fbbf48e76070f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:35.742873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '136d49c60f274d95ac8fbbf48e76070f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:35.743503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '136d49c60f274d95ac8fbbf48e76070f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:35.757081 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '136d49c60f274d95ac8fbbf48e76070f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:35.771935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1003/2012] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:35 2022] DELETE /identity/v3/users/136d49c60f274d95ac8fbbf48e76070f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:35.776247 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566877b7-eacc-45a7-8a4c-e07ddd44d6e4 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:35.776601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566877b7-eacc-45a7-8a4c-e07ddd44d6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:35.776795 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566877b7-eacc-45a7-8a4c-e07ddd44d6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:35.797333 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-566877b7-eacc-45a7-8a4c-e07ddd44d6e4 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:35.798527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-566877b7-eacc-45a7-8a4c-e07ddd44d6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:35.844472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:35.860257 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1010/2013] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:35.870073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:35.871560 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:35.930324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110bd9a-140e-4d59-b47e-9f83d41dcb4e admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:35.935118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e06631-7d55-4d6d-bb45-faec066a329f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:35.935731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e06631-7d55-4d6d-bb45-faec066a329f service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:35.936021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e06631-7d55-4d6d-bb45-faec066a329f service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:35.936299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e06631-7d55-4d6d-bb45-faec066a329f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:35.946358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e06631-7d55-4d6d-bb45-faec066a329f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:35.947794 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e06631-7d55-4d6d-bb45-faec066a329f service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:35.958306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1004/2014] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:36.339711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566877b7-eacc-45a7-8a4c-e07ddd44d6e4 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:36.341348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-566877b7-eacc-45a7-8a4c-e07ddd44d6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:36.377407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566877b7-eacc-45a7-8a4c-e07ddd44d6e4 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:36.391062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:36.392737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:36.393526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:36.394160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] PATH_INFO: `/v3/projects/b1a53ba49be8464d9bcdbfb3c74b6061` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:36.397664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1a53ba49be8464d9bcdbfb3c74b6061)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:36.401908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:36.482056 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1a53ba49be8464d9bcdbfb3c74b6061', 'request_id': 'req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.505554 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1a53ba49be8464d9bcdbfb3c74b6061', 'request_id': 'req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.516027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1a53ba49be8464d9bcdbfb3c74b6061', 'request_id': 'req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.537442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] The token cache is being invalidate because project b1a53ba49be8464d9bcdbfb3c74b6061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:36.537758 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.538444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.540781 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1011/2015] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:36 2022] DELETE /identity/v3/projects/b1a53ba49be8464d9bcdbfb3c74b6061 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:36.544511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e06631-7d55-4d6d-bb45-faec066a329f service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:36.562615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e06631-7d55-4d6d-bb45-faec066a329f service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:36.615951 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:36.639292 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41280933-d43b-4e90-8a23-5501277dc700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:36.639948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41280933-d43b-4e90-8a23-5501277dc700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:36.640261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41280933-d43b-4e90-8a23-5501277dc700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:36.640534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41280933-d43b-4e90-8a23-5501277dc700 admin admin] PATH_INFO: `/v3/users/45252f6ff97c4a669159929459849647` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:36.641690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41280933-d43b-4e90-8a23-5501277dc700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45252f6ff97c4a669159929459849647)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:36.643711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41280933-d43b-4e90-8a23-5501277dc700 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:36.705614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-41280933-d43b-4e90-8a23-5501277dc700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45252f6ff97c4a669159929459849647'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.716818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-41280933-d43b-4e90-8a23-5501277dc700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45252f6ff97c4a669159929459849647'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.718011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-41280933-d43b-4e90-8a23-5501277dc700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45252f6ff97c4a669159929459849647'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.718708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-41280933-d43b-4e90-8a23-5501277dc700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45252f6ff97c4a669159929459849647'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.736807 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-41280933-d43b-4e90-8a23-5501277dc700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45252f6ff97c4a669159929459849647'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.750908 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1005/2016] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:36 2022] DELETE /identity/v3/users/45252f6ff97c4a669159929459849647 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:37.057351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.058814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:37.073027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76fe5-e885-4b9f-b2e0-a8945dcf509b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.132403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.133050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.133329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.133600 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9 admin admin] PATH_INFO: `/v3/projects/de20cc5e2c8848b78dbb5af6ad0ab39c` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.134730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de20cc5e2c8848b78dbb5af6ad0ab39c)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.136709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.185425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de20cc5e2c8848b78dbb5af6ad0ab39c', 'request_id': 'req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:37.195396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de20cc5e2c8848b78dbb5af6ad0ab39c', 'request_id': 'req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:37.202297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de20cc5e2c8848b78dbb5af6ad0ab39c', 'request_id': 'req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:37.215564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9 admin admin] The token cache is being invalidate because project de20cc5e2c8848b78dbb5af6ad0ab39c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:37.215853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:37.216569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4919f77e-0754-4dd9-a9ed-b21ebfe287a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:37.218064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1012/2017] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:37 2022] DELETE /identity/v3/projects/de20cc5e2c8848b78dbb5af6ad0ab39c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:37.223656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9e3ce-1d71-45f9-8334-2c821bc04c0f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.223942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9e3ce-1d71-45f9-8334-2c821bc04c0f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.224247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9e3ce-1d71-45f9-8334-2c821bc04c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.245963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-9de9e3ce-1d71-45f9-8334-2c821bc04c0f None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:37.247176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9de9e3ce-1d71-45f9-8334-2c821bc04c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:37.310605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1006/2018] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:37.319595 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037225b-9054-43e1-9ef2-508611c03eb6 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.320402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037225b-9054-43e1-9ef2-508611c03eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.320402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037225b-9054-43e1-9ef2-508611c03eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.329289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:37.345056 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6037225b-9054-43e1-9ef2-508611c03eb6 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:37.346263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6037225b-9054-43e1-9ef2-508611c03eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:37.355243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1013/2019] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:37.359484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9e3ce-1d71-45f9-8334-2c821bc04c0f None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.369462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9e3ce-1d71-45f9-8334-2c821bc04c0f None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.374068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2de598-c0e4-456b-ba3d-c7701f65d79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.374680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2de598-c0e4-456b-ba3d-c7701f65d79f admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.375210 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2de598-c0e4-456b-ba3d-c7701f65d79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.375484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2de598-c0e4-456b-ba3d-c7701f65d79f admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.376316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2de598-c0e4-456b-ba3d-c7701f65d79f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:37.376478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2de598-c0e4-456b-ba3d-c7701f65d79f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.378279 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2de598-c0e4-456b-ba3d-c7701f65d79f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.395314 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1007/2020] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:37.397891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037225b-9054-43e1-9ef2-508611c03eb6 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.412180 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037225b-9054-43e1-9ef2-508611c03eb6 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.417285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166311db-2d76-4a2a-9af2-6e146834e661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.417907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166311db-2d76-4a2a-9af2-6e146834e661 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.418504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166311db-2d76-4a2a-9af2-6e146834e661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.418790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166311db-2d76-4a2a-9af2-6e146834e661 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.419427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166311db-2d76-4a2a-9af2-6e146834e661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.421368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166311db-2d76-4a2a-9af2-6e146834e661 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.456483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1014/2021] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:37 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:37.459409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2de598-c0e4-456b-ba3d-c7701f65d79f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.470945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2de598-c0e4-456b-ba3d-c7701f65d79f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.524934 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:37.540855 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49b8cc-3cb9-406c-8952-86614b9df708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.542430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49b8cc-3cb9-406c-8952-86614b9df708 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.542849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49b8cc-3cb9-406c-8952-86614b9df708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.543157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49b8cc-3cb9-406c-8952-86614b9df708 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.544429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49b8cc-3cb9-406c-8952-86614b9df708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.546306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49b8cc-3cb9-406c-8952-86614b9df708 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.568916 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1008/2022] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:37 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:37.573328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166311db-2d76-4a2a-9af2-6e146834e661 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.584428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166311db-2d76-4a2a-9af2-6e146834e661 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.590611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c49d94-3668-494e-9e62-996de537f26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.591233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c49d94-3668-494e-9e62-996de537f26b admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.591507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c49d94-3668-494e-9e62-996de537f26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.591784 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c49d94-3668-494e-9e62-996de537f26b admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.592552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c49d94-3668-494e-9e62-996de537f26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.594796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c49d94-3668-494e-9e62-996de537f26b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.610366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1015/2023] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:37.611581 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49b8cc-3cb9-406c-8952-86614b9df708 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.622354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49b8cc-3cb9-406c-8952-86614b9df708 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.627801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de58cc3-a155-4943-bb73-9375e307fb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.628534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de58cc3-a155-4943-bb73-9375e307fb25 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.629323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de58cc3-a155-4943-bb73-9375e307fb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.630231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de58cc3-a155-4943-bb73-9375e307fb25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.631007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de58cc3-a155-4943-bb73-9375e307fb25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.632550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de58cc3-a155-4943-bb73-9375e307fb25 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.648598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1009/2024] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:37.655947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c49d94-3668-494e-9e62-996de537f26b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.667246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c49d94-3668-494e-9e62-996de537f26b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.672070 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9142cc5-fedf-4022-8b9b-11449d1b32f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.672751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9142cc5-fedf-4022-8b9b-11449d1b32f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.673044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9142cc5-fedf-4022-8b9b-11449d1b32f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.673308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9142cc5-fedf-4022-8b9b-11449d1b32f7 admin admin] PATH_INFO: `/v3/projects/958c56742a854fb88237f4ebc5ef5749/users/8183e1619775451283cb6fec3935243b/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.693867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9142cc5-fedf-4022-8b9b-11449d1b32f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=958c56742a854fb88237f4ebc5ef5749, user_id=8183e1619775451283cb6fec3935243b, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.696632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9142cc5-fedf-4022-8b9b-11449d1b32f7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.713945 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1016/2025] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:37 2022] PUT /identity/v3/projects/958c56742a854fb88237f4ebc5ef5749/users/8183e1619775451283cb6fec3935243b/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:37.725721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c54bd-0f78-4060-b79b-b8cf6f98dc71 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.725721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c54bd-0f78-4060-b79b-b8cf6f98dc71 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.725721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c54bd-0f78-4060-b79b-b8cf6f98dc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.756861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-b09c54bd-0f78-4060-b79b-b8cf6f98dc71 None None] MFA Rules not processed for user `8183e1619775451283cb6fec3935243b`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:37.758103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b09c54bd-0f78-4060-b79b-b8cf6f98dc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:37.846542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1010/2026] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:37.858569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9142cc5-fedf-4022-8b9b-11449d1b32f7 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.860458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c9142cc5-fedf-4022-8b9b-11449d1b32f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:37.962817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9142cc5-fedf-4022-8b9b-11449d1b32f7 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.966834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7514f-b509-4b00-9780-0515a76db8df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c9d444b3294ed18c3d488cf0a3eeb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.967480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7514f-b509-4b00-9780-0515a76db8df service glance] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.967781 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7514f-b509-4b00-9780-0515a76db8df service glance] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.968173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7514f-b509-4b00-9780-0515a76db8df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.977827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7514f-b509-4b00-9780-0515a76db8df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.979299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7514f-b509-4b00-9780-0515a76db8df service glance] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.990871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1017/2027] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:42.943332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c54bd-0f78-4060-b79b-b8cf6f98dc71 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:42.944886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b09c54bd-0f78-4060-b79b-b8cf6f98dc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:43.021167 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c54bd-0f78-4060-b79b-b8cf6f98dc71 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:43.025763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:43.026414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.026696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.026971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] PATH_INFO: `/v3/users/93d9aad7dd1f43e0ab264ab5157ae845` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.028216 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93d9aad7dd1f43e0ab264ab5157ae845)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:43.030236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:43.083241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93d9aad7dd1f43e0ab264ab5157ae845'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.097034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93d9aad7dd1f43e0ab264ab5157ae845'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.098176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93d9aad7dd1f43e0ab264ab5157ae845'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.098779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93d9aad7dd1f43e0ab264ab5157ae845'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.100779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:43.112781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93d9aad7dd1f43e0ab264ab5157ae845'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.124784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1011/2028] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:42 2022] DELETE /identity/v3/users/93d9aad7dd1f43e0ab264ab5157ae845 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:43.129539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf928e4-938f-4718-ae2e-3f951c32c487 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.130053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf928e4-938f-4718-ae2e-3f951c32c487 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.130146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf928e4-938f-4718-ae2e-3f951c32c487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.150071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ecf928e4-938f-4718-ae2e-3f951c32c487 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:43.151264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ecf928e4-938f-4718-ae2e-3f951c32c487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:43.211553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1018/2029] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:43.221127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:43.223256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:43.279899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb6785-77c8-40e1-aeed-ffb1ece9e4a6 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:43.283757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b37a0-4eff-47df-a7cb-d7b579a9c2c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:43.285019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b37a0-4eff-47df-a7cb-d7b579a9c2c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.285296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b37a0-4eff-47df-a7cb-d7b579a9c2c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.285619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b37a0-4eff-47df-a7cb-d7b579a9c2c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.294572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b37a0-4eff-47df-a7cb-d7b579a9c2c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:43.296018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b37a0-4eff-47df-a7cb-d7b579a9c2c1 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:43.305444 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1012/2030] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:43.608874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf928e4-938f-4718-ae2e-3f951c32c487 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:43.610379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ecf928e4-938f-4718-ae2e-3f951c32c487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:43.625257 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf928e4-938f-4718-ae2e-3f951c32c487 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:43.630706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:43.631359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.631705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.631984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] PATH_INFO: `/v3/projects/d485e5cd429d444f9435a12f001d8109` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.633191 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d485e5cd429d444f9435a12f001d8109)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:43.635187 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:43.666506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:43.695615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd485e5cd429d444f9435a12f001d8109', 'request_id': 'req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.705183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd485e5cd429d444f9435a12f001d8109', 'request_id': 'req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.713735 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd485e5cd429d444f9435a12f001d8109', 'request_id': 'req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.726910 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] The token cache is being invalidate because project d485e5cd429d444f9435a12f001d8109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:43.727267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.728064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.729710 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1019/2031] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:43 2022] DELETE /identity/v3/projects/d485e5cd429d444f9435a12f001d8109 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:43.733872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b37a0-4eff-47df-a7cb-d7b579a9c2c1 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:43.744122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b37a0-4eff-47df-a7cb-d7b579a9c2c1 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:43.816402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2398-a0ba-47da-820a-7fc3dfe56796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:43.818131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2398-a0ba-47da-820a-7fc3dfe56796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.818499 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2398-a0ba-47da-820a-7fc3dfe56796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.818915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2398-a0ba-47da-820a-7fc3dfe56796 admin admin] PATH_INFO: `/v3/users/d70a6b721dfc4cea8990c3ac45942576` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.820266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2398-a0ba-47da-820a-7fc3dfe56796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d70a6b721dfc4cea8990c3ac45942576)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:43.822258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2398-a0ba-47da-820a-7fc3dfe56796 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:43.879005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f08f2398-a0ba-47da-820a-7fc3dfe56796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd70a6b721dfc4cea8990c3ac45942576'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.888967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f08f2398-a0ba-47da-820a-7fc3dfe56796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd70a6b721dfc4cea8990c3ac45942576'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.889906 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f08f2398-a0ba-47da-820a-7fc3dfe56796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd70a6b721dfc4cea8990c3ac45942576'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.890518 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f08f2398-a0ba-47da-820a-7fc3dfe56796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd70a6b721dfc4cea8990c3ac45942576'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.909937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f08f2398-a0ba-47da-820a-7fc3dfe56796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd70a6b721dfc4cea8990c3ac45942576'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:43.926128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:43.926128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1013/2032] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:43 2022] DELETE /identity/v3/users/d70a6b721dfc4cea8990c3ac45942576 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:43.935138 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:43.936747 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:43.951204 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c3b69-5bcf-4e20-b43e-cf466d40fe78 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.049883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.051159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.051433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.051711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] PATH_INFO: `/v3/users/8ad9c402520043bd8373f29790564971` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.052853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ad9c402520043bd8373f29790564971)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.054768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.124568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ad9c402520043bd8373f29790564971'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.146748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ad9c402520043bd8373f29790564971'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.158157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ad9c402520043bd8373f29790564971'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.170401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ad9c402520043bd8373f29790564971'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.171043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ad9c402520043bd8373f29790564971'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.178274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1020/2033] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:43 2022] DELETE /identity/v3/users/8ad9c402520043bd8373f29790564971 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:44.183115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d053d-a7ab-4e9d-8526-0922ec3dac80 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.183901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d053d-a7ab-4e9d-8526-0922ec3dac80 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.183901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d053d-a7ab-4e9d-8526-0922ec3dac80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.204092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-079d053d-a7ab-4e9d-8526-0922ec3dac80 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:44.205437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-079d053d-a7ab-4e9d-8526-0922ec3dac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:44.259081 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.260756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:44.279271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1014/2034] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:44.290289 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d053d-a7ab-4e9d-8526-0922ec3dac80 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.291877 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-079d053d-a7ab-4e9d-8526-0922ec3dac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:44.301440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:44.349314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f4498-3d80-49d4-a787-a3c2092ee6b0 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.354140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbaf11f-a746-4e96-8046-654e5d1ccc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.354825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbaf11f-a746-4e96-8046-654e5d1ccc17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.355089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbaf11f-a746-4e96-8046-654e5d1ccc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.355357 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbaf11f-a746-4e96-8046-654e5d1ccc17 admin admin] PATH_INFO: `/v3/projects/db52a8ec32e148068c370876e8d3af28` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.356970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbaf11f-a746-4e96-8046-654e5d1ccc17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db52a8ec32e148068c370876e8d3af28)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.357558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d053d-a7ab-4e9d-8526-0922ec3dac80 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.358616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbaf11f-a746-4e96-8046-654e5d1ccc17 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.361485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6198c-e52d-4bcf-a7a4-dd52a21f197d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.362109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6198c-e52d-4bcf-a7a4-dd52a21f197d service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.362396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6198c-e52d-4bcf-a7a4-dd52a21f197d service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.362671 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6198c-e52d-4bcf-a7a4-dd52a21f197d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.372552 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6198c-e52d-4bcf-a7a4-dd52a21f197d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.373981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6198c-e52d-4bcf-a7a4-dd52a21f197d service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.385556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1015/2035] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:44.413032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9fbaf11f-a746-4e96-8046-654e5d1ccc17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db52a8ec32e148068c370876e8d3af28', 'request_id': 'req-9fbaf11f-a746-4e96-8046-654e5d1ccc17'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.420870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9fbaf11f-a746-4e96-8046-654e5d1ccc17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db52a8ec32e148068c370876e8d3af28', 'request_id': 'req-9fbaf11f-a746-4e96-8046-654e5d1ccc17'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.429324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9fbaf11f-a746-4e96-8046-654e5d1ccc17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db52a8ec32e148068c370876e8d3af28', 'request_id': 'req-9fbaf11f-a746-4e96-8046-654e5d1ccc17'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.441051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9fbaf11f-a746-4e96-8046-654e5d1ccc17 admin admin] The token cache is being invalidate because project db52a8ec32e148068c370876e8d3af28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:44.441340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9fbaf11f-a746-4e96-8046-654e5d1ccc17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.442436 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9fbaf11f-a746-4e96-8046-654e5d1ccc17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.449259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1021/2036] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:44 2022] DELETE /identity/v3/projects/db52a8ec32e148068c370876e8d3af28 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:44.452285 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bbd9f-1e8b-431b-8055-b53260257c4b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.452712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bbd9f-1e8b-431b-8055-b53260257c4b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.453047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bbd9f-1e8b-431b-8055-b53260257c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.476657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-4c3bbd9f-1e8b-431b-8055-b53260257c4b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:44.477869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4c3bbd9f-1e8b-431b-8055-b53260257c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:44.540954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1016/2037] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:44.549696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d389b6a-cf1c-4ac0-9268-6015ee91c0c3 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.550549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d389b6a-cf1c-4ac0-9268-6015ee91c0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.550812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d389b6a-cf1c-4ac0-9268-6015ee91c0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.574077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-8d389b6a-cf1c-4ac0-9268-6015ee91c0c3 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:44.575458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8d389b6a-cf1c-4ac0-9268-6015ee91c0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:44.583719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1022/2038] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:44.589015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bbd9f-1e8b-431b-8055-b53260257c4b None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.600231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bbd9f-1e8b-431b-8055-b53260257c4b None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.605065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1e8f3-4d04-46ba-9b63-459a39d09439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.605770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1e8f3-4d04-46ba-9b63-459a39d09439 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.606114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1e8f3-4d04-46ba-9b63-459a39d09439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.606448 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1e8f3-4d04-46ba-9b63-459a39d09439 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.607164 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1e8f3-4d04-46ba-9b63-459a39d09439 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:44.607533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1e8f3-4d04-46ba-9b63-459a39d09439 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.609250 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1e8f3-4d04-46ba-9b63-459a39d09439 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.622684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1017/2039] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:44.627352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d389b6a-cf1c-4ac0-9268-6015ee91c0c3 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.638362 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d389b6a-cf1c-4ac0-9268-6015ee91c0c3 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.643095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799abac4-7b67-4c6d-b773-146d7c458bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.643750 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799abac4-7b67-4c6d-b773-146d7c458bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.644058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799abac4-7b67-4c6d-b773-146d7c458bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.644335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799abac4-7b67-4c6d-b773-146d7c458bb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.644993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799abac4-7b67-4c6d-b773-146d7c458bb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.646900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799abac4-7b67-4c6d-b773-146d7c458bb7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.676602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1e8f3-4d04-46ba-9b63-459a39d09439 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.678771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-06d1e8f3-4d04-46ba-9b63-459a39d09439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:44.680893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1023/2040] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:44 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:44.685208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799abac4-7b67-4c6d-b773-146d7c458bb7 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.705114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1e8f3-4d04-46ba-9b63-459a39d09439 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.730308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:44.764630 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.765036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799abac4-7b67-4c6d-b773-146d7c458bb7 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.765321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.765602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.765911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] PATH_INFO: `/v3/projects/b28386de5d164596b344b05f0d71848e` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.767050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b28386de5d164596b344b05f0d71848e)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.769150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.770171 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad78d57-192f-4ca6-af99-c03cb7022d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.770785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad78d57-192f-4ca6-af99-c03cb7022d6c admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.771059 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad78d57-192f-4ca6-af99-c03cb7022d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.771326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad78d57-192f-4ca6-af99-c03cb7022d6c admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.771974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad78d57-192f-4ca6-af99-c03cb7022d6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.773925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad78d57-192f-4ca6-af99-c03cb7022d6c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.796109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1024/2041] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:44 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:44.800794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad78d57-192f-4ca6-af99-c03cb7022d6c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.811376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad78d57-192f-4ca6-af99-c03cb7022d6c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.821431 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b28386de5d164596b344b05f0d71848e', 'request_id': 'req-472342bb-0114-478d-b32b-ed3b3028a4d9'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.824928 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:44.828039 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b28386de5d164596b344b05f0d71848e', 'request_id': 'req-472342bb-0114-478d-b32b-ed3b3028a4d9'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.837451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b28386de5d164596b344b05f0d71848e', 'request_id': 'req-472342bb-0114-478d-b32b-ed3b3028a4d9'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.853048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] The token cache is being invalidate because project b28386de5d164596b344b05f0d71848e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:44.853469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.854135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:44.856055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1018/2042] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:44 2022] DELETE /identity/v3/projects/b28386de5d164596b344b05f0d71848e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:44.859970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.861526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:44.879670 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d094341-94d3-4820-899d-7934f8a25e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.880326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d094341-94d3-4820-899d-7934f8a25e83 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.880635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d094341-94d3-4820-899d-7934f8a25e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.880878 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d094341-94d3-4820-899d-7934f8a25e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.881624 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d094341-94d3-4820-899d-7934f8a25e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.883157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d094341-94d3-4820-899d-7934f8a25e83 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.896701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1025/2043] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:44.898010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d094341-94d3-4820-899d-7934f8a25e83 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.900438 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6d094341-94d3-4820-899d-7934f8a25e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:44.913956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d094341-94d3-4820-899d-7934f8a25e83 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.918719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210f5fa-b286-40ba-a226-fc58705c8581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.919315 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210f5fa-b286-40ba-a226-fc58705c8581 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.919595 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210f5fa-b286-40ba-a226-fc58705c8581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.919938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210f5fa-b286-40ba-a226-fc58705c8581 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.920623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210f5fa-b286-40ba-a226-fc58705c8581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.922241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210f5fa-b286-40ba-a226-fc58705c8581 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.932651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472342bb-0114-478d-b32b-ed3b3028a4d9 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.932961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1026/2044] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:44.937150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210f5fa-b286-40ba-a226-fc58705c8581 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.937419 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.938052 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.938328 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.938597 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] PATH_INFO: `/v3/users/ac7eceff6b864a208191b25497fe457e` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.939712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac7eceff6b864a208191b25497fe457e)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.942490 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.947869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210f5fa-b286-40ba-a226-fc58705c8581 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.952613 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5549d4-bb8d-4364-8d29-18c6b6fc5728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.953297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5549d4-bb8d-4364-8d29-18c6b6fc5728 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.953535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5549d4-bb8d-4364-8d29-18c6b6fc5728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.953809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5549d4-bb8d-4364-8d29-18c6b6fc5728 admin admin] PATH_INFO: `/v3/projects/585255dc2f0344dbbad821bf1110cda5/users/1d34f688d0b94401a1907e9845a61fc5/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.969894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5549d4-bb8d-4364-8d29-18c6b6fc5728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=585255dc2f0344dbbad821bf1110cda5, user_id=1d34f688d0b94401a1907e9845a61fc5, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.973043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5549d4-bb8d-4364-8d29-18c6b6fc5728 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.986919 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1027/2045] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:44 2022] PUT /identity/v3/projects/585255dc2f0344dbbad821bf1110cda5/users/1d34f688d0b94401a1907e9845a61fc5/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:44.993271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac7eceff6b864a208191b25497fe457e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:45.000271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9342922-591f-4bdc-8b57-c3e55cd8153f None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:45.000542 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9342922-591f-4bdc-8b57-c3e55cd8153f None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:45.000853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9342922-591f-4bdc-8b57-c3e55cd8153f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:45.010785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac7eceff6b864a208191b25497fe457e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:45.011458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac7eceff6b864a208191b25497fe457e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:45.012120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac7eceff6b864a208191b25497fe457e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:45.017833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:45.031835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac7eceff6b864a208191b25497fe457e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:45.032996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-a9342922-591f-4bdc-8b57-c3e55cd8153f None None] MFA Rules not processed for user `1d34f688d0b94401a1907e9845a61fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:45.034350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a9342922-591f-4bdc-8b57-c3e55cd8153f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:45.055730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1019/2046] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:44 2022] DELETE /identity/v3/users/ac7eceff6b864a208191b25497fe457e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:45.123496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1028/2047] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:45.136999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:45.138939 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:45.232095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a27fc7-d605-4734-af94-b2b68d0afc45 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:45.237029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2dfd46-d168-4423-b00a-c71c6733ea68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:45.237660 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dfd46-d168-4423-b00a-c71c6733ea68 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:45.238066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dfd46-d168-4423-b00a-c71c6733ea68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:45.238498 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2dfd46-d168-4423-b00a-c71c6733ea68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:45.312274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2dfd46-d168-4423-b00a-c71c6733ea68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:45.313920 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2dfd46-d168-4423-b00a-c71c6733ea68 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:45.327727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1020/2048] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:45.400657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9342922-591f-4bdc-8b57-c3e55cd8153f None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:45.402010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a9342922-591f-4bdc-8b57-c3e55cd8153f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:45.508797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9342922-591f-4bdc-8b57-c3e55cd8153f None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:45.533500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:45.587636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56521c67-6c08-4bea-b511-0f2d631d8eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:45.588486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56521c67-6c08-4bea-b511-0f2d631d8eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:45.588800 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56521c67-6c08-4bea-b511-0f2d631d8eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:45.589107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56521c67-6c08-4bea-b511-0f2d631d8eef admin admin] PATH_INFO: `/v3/projects/32a9b1304bf94b0689359c20d1e14f20` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:45.590795 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56521c67-6c08-4bea-b511-0f2d631d8eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=32a9b1304bf94b0689359c20d1e14f20)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:45.593545 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56521c67-6c08-4bea-b511-0f2d631d8eef admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:45.671264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-56521c67-6c08-4bea-b511-0f2d631d8eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32a9b1304bf94b0689359c20d1e14f20', 'request_id': 'req-56521c67-6c08-4bea-b511-0f2d631d8eef'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:45.682299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-56521c67-6c08-4bea-b511-0f2d631d8eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32a9b1304bf94b0689359c20d1e14f20', 'request_id': 'req-56521c67-6c08-4bea-b511-0f2d631d8eef'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:45.692435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-56521c67-6c08-4bea-b511-0f2d631d8eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32a9b1304bf94b0689359c20d1e14f20', 'request_id': 'req-56521c67-6c08-4bea-b511-0f2d631d8eef'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:45.707216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-56521c67-6c08-4bea-b511-0f2d631d8eef admin admin] The token cache is being invalidate because project 32a9b1304bf94b0689359c20d1e14f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:45.707594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-56521c67-6c08-4bea-b511-0f2d631d8eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:45.708311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-56521c67-6c08-4bea-b511-0f2d631d8eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:45.710445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1029/2049] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:45 2022] DELETE /identity/v3/projects/32a9b1304bf94b0689359c20d1e14f20 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:45.714919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaac7d-1952-40bc-acb1-c8db27256f56 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:45.715307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaac7d-1952-40bc-acb1-c8db27256f56 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:45.715653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaac7d-1952-40bc-acb1-c8db27256f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:45.741604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-fccaac7d-1952-40bc-acb1-c8db27256f56 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:45.742877 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-fccaac7d-1952-40bc-acb1-c8db27256f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:45.806319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1021/2050] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:45.814259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fe643-a052-48d4-9b62-8395424eb660 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:45.814538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fe643-a052-48d4-9b62-8395424eb660 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:45.814807 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fe643-a052-48d4-9b62-8395424eb660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:45.834241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-680fe643-a052-48d4-9b62-8395424eb660 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:45.835340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-680fe643-a052-48d4-9b62-8395424eb660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:45.843066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1030/2051] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:45.847932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaac7d-1952-40bc-acb1-c8db27256f56 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:45.857647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaac7d-1952-40bc-acb1-c8db27256f56 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:45.862978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0f446-1a99-4ca3-834e-62470e46f595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:45.863601 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0f446-1a99-4ca3-834e-62470e46f595 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:45.863876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0f446-1a99-4ca3-834e-62470e46f595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:45.864182 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0f446-1a99-4ca3-834e-62470e46f595 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:45.864830 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0f446-1a99-4ca3-834e-62470e46f595 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:45.865136 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0f446-1a99-4ca3-834e-62470e46f595 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:45.866742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0f446-1a99-4ca3-834e-62470e46f595 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:45.878714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1022/2052] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:45.883172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fe643-a052-48d4-9b62-8395424eb660 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:45.893238 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fe643-a052-48d4-9b62-8395424eb660 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:45.898526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd544525-9412-4d08-a2f8-255cfb968bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:45.899173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd544525-9412-4d08-a2f8-255cfb968bac admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:45.899448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd544525-9412-4d08-a2f8-255cfb968bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:45.899717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd544525-9412-4d08-a2f8-255cfb968bac admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:45.900404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd544525-9412-4d08-a2f8-255cfb968bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:45.902356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd544525-9412-4d08-a2f8-255cfb968bac admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:45.931479 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1031/2053] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:45 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:45.939988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0f446-1a99-4ca3-834e-62470e46f595 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:45.950256 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0f446-1a99-4ca3-834e-62470e46f595 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:45.963349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:46.011087 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72bedd-c4d0-41bb-be35-7915c51c6583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:46.011699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72bedd-c4d0-41bb-be35-7915c51c6583 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.011979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72bedd-c4d0-41bb-be35-7915c51c6583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.012284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72bedd-c4d0-41bb-be35-7915c51c6583 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.013186 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72bedd-c4d0-41bb-be35-7915c51c6583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:46.015106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72bedd-c4d0-41bb-be35-7915c51c6583 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:46.038926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1023/2054] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:45 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:46.043637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd544525-9412-4d08-a2f8-255cfb968bac admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:46.054706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd544525-9412-4d08-a2f8-255cfb968bac admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:46.059382 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b288f58-1b61-457f-a32b-77ac435bee3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:46.059974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b288f58-1b61-457f-a32b-77ac435bee3c admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.060281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b288f58-1b61-457f-a32b-77ac435bee3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.060557 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b288f58-1b61-457f-a32b-77ac435bee3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.061197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b288f58-1b61-457f-a32b-77ac435bee3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:46.062857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b288f58-1b61-457f-a32b-77ac435bee3c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:46.073913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1032/2055] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:46.078171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72bedd-c4d0-41bb-be35-7915c51c6583 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:46.089453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72bedd-c4d0-41bb-be35-7915c51c6583 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:46.094584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b734a-4053-404f-9fe6-0e9e4fbda620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:46.095208 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b734a-4053-404f-9fe6-0e9e4fbda620 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.095484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b734a-4053-404f-9fe6-0e9e4fbda620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.095895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b734a-4053-404f-9fe6-0e9e4fbda620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.096583 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b734a-4053-404f-9fe6-0e9e4fbda620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:46.098294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b734a-4053-404f-9fe6-0e9e4fbda620 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:46.109440 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1024/2056] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:46.114413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b288f58-1b61-457f-a32b-77ac435bee3c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:46.124987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b288f58-1b61-457f-a32b-77ac435bee3c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:46.130747 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327b4f9-f768-40b1-bdce-9510ee6bc9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:46.131345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327b4f9-f768-40b1-bdce-9510ee6bc9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.131619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327b4f9-f768-40b1-bdce-9510ee6bc9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.131887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327b4f9-f768-40b1-bdce-9510ee6bc9f9 admin admin] PATH_INFO: `/v3/projects/ef940fad71114e0a80124e1829353bd5/users/77791e9ca3f74af08b74ddf175bfca6a/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.150879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327b4f9-f768-40b1-bdce-9510ee6bc9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef940fad71114e0a80124e1829353bd5, user_id=77791e9ca3f74af08b74ddf175bfca6a, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:46.153612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327b4f9-f768-40b1-bdce-9510ee6bc9f9 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:46.177082 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1033/2057] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:46 2022] PUT /identity/v3/projects/ef940fad71114e0a80124e1829353bd5/users/77791e9ca3f74af08b74ddf175bfca6a/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:46.183447 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6502b7b-295b-4da8-bf94-7f1e58dd8880 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.183810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6502b7b-295b-4da8-bf94-7f1e58dd8880 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.184167 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6502b7b-295b-4da8-bf94-7f1e58dd8880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.220014 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c6502b7b-295b-4da8-bf94-7f1e58dd8880 None None] MFA Rules not processed for user `77791e9ca3f74af08b74ddf175bfca6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:46.222038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c6502b7b-295b-4da8-bf94-7f1e58dd8880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:46.327679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1025/2058] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:46.332608 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327b4f9-f768-40b1-bdce-9510ee6bc9f9 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:46.389698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:46.412606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327b4f9-f768-40b1-bdce-9510ee6bc9f9 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:46.420022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec22cf-9204-4dd8-9a32-0042a2aef6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:46.420725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec22cf-9204-4dd8-9a32-0042a2aef6db admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.421004 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec22cf-9204-4dd8-9a32-0042a2aef6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.421278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec22cf-9204-4dd8-9a32-0042a2aef6db admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.421933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec22cf-9204-4dd8-9a32-0042a2aef6db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:46.424149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec22cf-9204-4dd8-9a32-0042a2aef6db admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:46.454496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1034/2059] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:46 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:46.459863 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6502b7b-295b-4da8-bf94-7f1e58dd8880 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:46.472696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6502b7b-295b-4da8-bf94-7f1e58dd8880 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:46.548672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890cf8e-3fb4-435a-ab20-fb22dbc27891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:46.549440 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890cf8e-3fb4-435a-ab20-fb22dbc27891 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.550604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890cf8e-3fb4-435a-ab20-fb22dbc27891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.550897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890cf8e-3fb4-435a-ab20-fb22dbc27891 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.551391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890cf8e-3fb4-435a-ab20-fb22dbc27891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:46.553487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890cf8e-3fb4-435a-ab20-fb22dbc27891 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:46.578005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1026/2060] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:46 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:46.582485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec22cf-9204-4dd8-9a32-0042a2aef6db admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:46.597594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec22cf-9204-4dd8-9a32-0042a2aef6db admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:46.603314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0fc02-f114-4509-a17b-292fc26778c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:46.603918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0fc02-f114-4509-a17b-292fc26778c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.604232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0fc02-f114-4509-a17b-292fc26778c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.604532 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0fc02-f114-4509-a17b-292fc26778c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.605336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0fc02-f114-4509-a17b-292fc26778c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:46.606975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0fc02-f114-4509-a17b-292fc26778c3 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:46.621068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1035/2061] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:46.628421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890cf8e-3fb4-435a-ab20-fb22dbc27891 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:46.648545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890cf8e-3fb4-435a-ab20-fb22dbc27891 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:46.654868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bde93-7043-414e-ad4a-964bbd673bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:46.655603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bde93-7043-414e-ad4a-964bbd673bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.655970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bde93-7043-414e-ad4a-964bbd673bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.656396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bde93-7043-414e-ad4a-964bbd673bd6 admin admin] PATH_INFO: `/v3/projects/7e934b7ced8546fdb8d3606ce3208e8f/users/c6a2c2455ac44bedb4bbf62bb11639bc/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.678305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:46.682976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bde93-7043-414e-ad4a-964bbd673bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e934b7ced8546fdb8d3606ce3208e8f, user_id=c6a2c2455ac44bedb4bbf62bb11639bc, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:46.687052 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bde93-7043-414e-ad4a-964bbd673bd6 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:46.703927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1027/2062] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:46 2022] PUT /identity/v3/projects/7e934b7ced8546fdb8d3606ce3208e8f/users/c6a2c2455ac44bedb4bbf62bb11639bc/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:46.709899 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0fc02-f114-4509-a17b-292fc26778c3 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:46.721853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0fc02-f114-4509-a17b-292fc26778c3 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:46.790562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cba0e-89e7-4567-979e-bd001e124355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:46.795053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cba0e-89e7-4567-979e-bd001e124355 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.795053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cba0e-89e7-4567-979e-bd001e124355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.795053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cba0e-89e7-4567-979e-bd001e124355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.795053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cba0e-89e7-4567-979e-bd001e124355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:46.795053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cba0e-89e7-4567-979e-bd001e124355 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:46.808038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1036/2063] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:46.813109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bde93-7043-414e-ad4a-964bbd673bd6 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:46.823916 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bde93-7043-414e-ad4a-964bbd673bd6 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:46.828671 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa557a3-4778-46f3-a3a0-dc14a04a719c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:46.829342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa557a3-4778-46f3-a3a0-dc14a04a719c admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.829621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa557a3-4778-46f3-a3a0-dc14a04a719c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.829895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa557a3-4778-46f3-a3a0-dc14a04a719c admin admin] PATH_INFO: `/v3/projects/7e934b7ced8546fdb8d3606ce3208e8f/users/c6a2c2455ac44bedb4bbf62bb11639bc/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.833036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa557a3-4778-46f3-a3a0-dc14a04a719c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e934b7ced8546fdb8d3606ce3208e8f, user_id=c6a2c2455ac44bedb4bbf62bb11639bc, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:46.835722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa557a3-4778-46f3-a3a0-dc14a04a719c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:46.853158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1028/2064] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:46 2022] PUT /identity/v3/projects/7e934b7ced8546fdb8d3606ce3208e8f/users/c6a2c2455ac44bedb4bbf62bb11639bc/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:46.865168 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515cd0c-2093-4903-b856-7bab9d1fa2dd None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.865370 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c19a2-4451-4e37-9517-e5a51a1863ad None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:46.865561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515cd0c-2093-4903-b856-7bab9d1fa2dd None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.865665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c19a2-4451-4e37-9517-e5a51a1863ad None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:46.865915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c19a2-4451-4e37-9517-e5a51a1863ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.866053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515cd0c-2093-4903-b856-7bab9d1fa2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:46.896842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-bc9c19a2-4451-4e37-9517-e5a51a1863ad None None] MFA Rules not processed for user `c6a2c2455ac44bedb4bbf62bb11639bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:46.898078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-bc9c19a2-4451-4e37-9517-e5a51a1863ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:46.899633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-8515cd0c-2093-4903-b856-7bab9d1fa2dd None None] MFA Rules not processed for user `1d34f688d0b94401a1907e9845a61fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:46.901694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8515cd0c-2093-4903-b856-7bab9d1fa2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:46.995988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1029/2065] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:47.002647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1037/2066] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:47.010589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c19a2-4451-4e37-9517-e5a51a1863ad None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:47.012405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-bc9c19a2-4451-4e37-9517-e5a51a1863ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:47.077645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c19a2-4451-4e37-9517-e5a51a1863ad None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:47.081819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101ce56-a244-4e45-b9fb-c6b23e47d25f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:47.083303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101ce56-a244-4e45-b9fb-c6b23e47d25f service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:47.083303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101ce56-a244-4e45-b9fb-c6b23e47d25f service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:47.083303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101ce56-a244-4e45-b9fb-c6b23e47d25f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:47.092903 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101ce56-a244-4e45-b9fb-c6b23e47d25f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:47.094334 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101ce56-a244-4e45-b9fb-c6b23e47d25f service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:47.104947 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1030/2067] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:47.203547 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515cd0c-2093-4903-b856-7bab9d1fa2dd None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:47.205388 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8515cd0c-2093-4903-b856-7bab9d1fa2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:47.225050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515cd0c-2093-4903-b856-7bab9d1fa2dd None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:47.229815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d806dfb-75cc-450f-a748-8c5127b32392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:47.229980 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d806dfb-75cc-450f-a748-8c5127b32392 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:47.230346 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d806dfb-75cc-450f-a748-8c5127b32392 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:47.230688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d806dfb-75cc-450f-a748-8c5127b32392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:47.311338 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d806dfb-75cc-450f-a748-8c5127b32392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:47.313697 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d806dfb-75cc-450f-a748-8c5127b32392 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:47.319773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:47.326797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1038/2068] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:47.368837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101ce56-a244-4e45-b9fb-c6b23e47d25f service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:47.370333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a101ce56-a244-4e45-b9fb-c6b23e47d25f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:47.501491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101ce56-a244-4e45-b9fb-c6b23e47d25f service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:47.513487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:47.513487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:47.513965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:47.514439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] PATH_INFO: `/v3/users/4aaeef813f704993a4d8951e9013d7dd` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:47.516110 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aaeef813f704993a4d8951e9013d7dd)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:47.520853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:47.528591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:47.615014 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aaeef813f704993a4d8951e9013d7dd'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:47.633497 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aaeef813f704993a4d8951e9013d7dd'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:47.635269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aaeef813f704993a4d8951e9013d7dd'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:47.638304 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aaeef813f704993a4d8951e9013d7dd'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:47.696814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aaeef813f704993a4d8951e9013d7dd'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:47.732956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1031/2069] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:47 2022] DELETE /identity/v3/users/4aaeef813f704993a4d8951e9013d7dd => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:47.754599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c145777-0890-4bab-b1e0-7572d1857c26 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:47.754599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c145777-0890-4bab-b1e0-7572d1857c26 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:47.754599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c145777-0890-4bab-b1e0-7572d1857c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:47.789987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-0c145777-0890-4bab-b1e0-7572d1857c26 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:47.792102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0c145777-0890-4bab-b1e0-7572d1857c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:47.935215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1039/2070] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:47.951352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:47.954169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:48.044954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51cf1c-649b-4de1-9924-9e425b6264b9 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:48.052530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae52944-3a7c-46e0-b94a-80e70177aac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:48.053460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52944-3a7c-46e0-b94a-80e70177aac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:48.053759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52944-3a7c-46e0-b94a-80e70177aac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:48.054131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52944-3a7c-46e0-b94a-80e70177aac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:48.068106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae52944-3a7c-46e0-b94a-80e70177aac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:48.080383 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae52944-3a7c-46e0-b94a-80e70177aac7 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:48.089647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1032/2071] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:48.471318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c145777-0890-4bab-b1e0-7572d1857c26 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:48.473610 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0c145777-0890-4bab-b1e0-7572d1857c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:48.489186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c145777-0890-4bab-b1e0-7572d1857c26 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:48.495413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:48.496346 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:48.496742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:48.497017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] PATH_INFO: `/v3/projects/e01fa2a12bff48b8b920872e4c6ae588` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:48.498663 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01fa2a12bff48b8b920872e4c6ae588)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:48.500692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:48.558167 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01fa2a12bff48b8b920872e4c6ae588', 'request_id': 'req-1514b74c-909d-45c1-8492-cc94435be893'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.568885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01fa2a12bff48b8b920872e4c6ae588', 'request_id': 'req-1514b74c-909d-45c1-8492-cc94435be893'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.578807 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01fa2a12bff48b8b920872e4c6ae588', 'request_id': 'req-1514b74c-909d-45c1-8492-cc94435be893'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.596334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] The token cache is being invalidate because project e01fa2a12bff48b8b920872e4c6ae588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:48.597296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.606780 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.609652 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1040/2072] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:48 2022] DELETE /identity/v3/projects/e01fa2a12bff48b8b920872e4c6ae588 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:48.615583 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae52944-3a7c-46e0-b94a-80e70177aac7 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:48.634389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae52944-3a7c-46e0-b94a-80e70177aac7 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:48.708585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:48.709305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:48.709572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:48.709903 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] PATH_INFO: `/v3/users/02b64cf20db64fc39e3ed67f6430dc8d` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:48.713486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b64cf20db64fc39e3ed67f6430dc8d)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:48.715308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:48.789375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:48.817494 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b64cf20db64fc39e3ed67f6430dc8d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.830364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b64cf20db64fc39e3ed67f6430dc8d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.831705 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b64cf20db64fc39e3ed67f6430dc8d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.832680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b64cf20db64fc39e3ed67f6430dc8d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.853604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b64cf20db64fc39e3ed67f6430dc8d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.899222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1033/2073] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:48 2022] DELETE /identity/v3/users/02b64cf20db64fc39e3ed67f6430dc8d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:49.330599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:49.332816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:49.346294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514b74c-909d-45c1-8492-cc94435be893 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:49.422451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:49.423092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:49.423372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:49.423648 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] PATH_INFO: `/v3/projects/cb88f173834a4513983bc7aef169dcd4` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:49.425497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb88f173834a4513983bc7aef169dcd4)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:49.427568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:49.451290 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:49.477013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb88f173834a4513983bc7aef169dcd4', 'request_id': 'req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.484400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb88f173834a4513983bc7aef169dcd4', 'request_id': 'req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.490871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb88f173834a4513983bc7aef169dcd4', 'request_id': 'req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.502002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] The token cache is being invalidate because project cb88f173834a4513983bc7aef169dcd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:49.502285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.503282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.504868 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1041/2074] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:49 2022] DELETE /identity/v3/projects/cb88f173834a4513983bc7aef169dcd4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:49.508949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:49.510344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:49.523661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a33012-d484-4560-917e-8f426aa7a8f2 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:49.580468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70898532-4d87-4f51-8011-530c4660cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:49.581180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70898532-4d87-4f51-8011-530c4660cd4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:49.581603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70898532-4d87-4f51-8011-530c4660cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:49.581882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70898532-4d87-4f51-8011-530c4660cd4a admin admin] PATH_INFO: `/v3/users/ff3d8c44932847e1ac8ab6d5dbf5c2cb` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:49.583069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70898532-4d87-4f51-8011-530c4660cd4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff3d8c44932847e1ac8ab6d5dbf5c2cb)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:49.585058 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70898532-4d87-4f51-8011-530c4660cd4a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:49.645065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-70898532-4d87-4f51-8011-530c4660cd4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff3d8c44932847e1ac8ab6d5dbf5c2cb'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.653842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-70898532-4d87-4f51-8011-530c4660cd4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff3d8c44932847e1ac8ab6d5dbf5c2cb'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.654752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-70898532-4d87-4f51-8011-530c4660cd4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff3d8c44932847e1ac8ab6d5dbf5c2cb'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.655482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-70898532-4d87-4f51-8011-530c4660cd4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff3d8c44932847e1ac8ab6d5dbf5c2cb'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.671507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-70898532-4d87-4f51-8011-530c4660cd4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff3d8c44932847e1ac8ab6d5dbf5c2cb'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.686817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1034/2075] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:49 2022] DELETE /identity/v3/users/ff3d8c44932847e1ac8ab6d5dbf5c2cb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:49.957340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:49.959849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:49.976678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7639b3-43c4-4b3f-bad0-b60fb86a60d8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.059598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.060282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.060569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.060838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d admin admin] PATH_INFO: `/v3/projects/19a0ccb72ee0460695346bdd507c5b2c` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.061983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a0ccb72ee0460695346bdd507c5b2c)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.063974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.113987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a0ccb72ee0460695346bdd507c5b2c', 'request_id': 'req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.121611 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a0ccb72ee0460695346bdd507c5b2c', 'request_id': 'req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.128551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a0ccb72ee0460695346bdd507c5b2c', 'request_id': 'req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.140913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d admin admin] The token cache is being invalidate because project 19a0ccb72ee0460695346bdd507c5b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:50.141193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.141900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4a0fd260-ea4a-4aec-b6a7-afebdc4d564d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.143930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1042/2076] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:49 2022] DELETE /identity/v3/projects/19a0ccb72ee0460695346bdd507c5b2c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:50.149061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0395005-093c-4bc9-badb-173d532bbac6 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.149350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0395005-093c-4bc9-badb-173d532bbac6 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.149631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0395005-093c-4bc9-badb-173d532bbac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.174029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-e0395005-093c-4bc9-badb-173d532bbac6 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:50.175368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e0395005-093c-4bc9-badb-173d532bbac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:50.198935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:50.261587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1035/2077] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:50.270032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a2540-e9ff-4082-ac3e-7d7924fb9c6e None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.270388 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a2540-e9ff-4082-ac3e-7d7924fb9c6e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.270732 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a2540-e9ff-4082-ac3e-7d7924fb9c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.306313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7e6a2540-e9ff-4082-ac3e-7d7924fb9c6e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:50.307567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7e6a2540-e9ff-4082-ac3e-7d7924fb9c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:50.317109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1043/2078] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:50.321415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0395005-093c-4bc9-badb-173d532bbac6 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.331725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0395005-093c-4bc9-badb-173d532bbac6 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.336809 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47002c1-af73-40f6-848c-a2ee003f0068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.337588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47002c1-af73-40f6-848c-a2ee003f0068 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.337831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47002c1-af73-40f6-848c-a2ee003f0068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.338199 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47002c1-af73-40f6-848c-a2ee003f0068 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.338919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47002c1-af73-40f6-848c-a2ee003f0068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:50.339296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47002c1-af73-40f6-848c-a2ee003f0068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.340974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47002c1-af73-40f6-848c-a2ee003f0068 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.359202 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1036/2079] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:50.364292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a2540-e9ff-4082-ac3e-7d7924fb9c6e None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.373904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a2540-e9ff-4082-ac3e-7d7924fb9c6e None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.380425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b358c-acf9-4885-ab75-2bdca4bb3fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.381063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b358c-acf9-4885-ab75-2bdca4bb3fca admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.381335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b358c-acf9-4885-ab75-2bdca4bb3fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.381604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b358c-acf9-4885-ab75-2bdca4bb3fca admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.382243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b358c-acf9-4885-ab75-2bdca4bb3fca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.384179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b358c-acf9-4885-ab75-2bdca4bb3fca admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.412980 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1044/2080] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:50 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:50.417197 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47002c1-af73-40f6-848c-a2ee003f0068 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.428948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47002c1-af73-40f6-848c-a2ee003f0068 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.495558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085d9a2-e1d2-4561-a238-01f8fbb0c0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.496060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085d9a2-e1d2-4561-a238-01f8fbb0c0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.496345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085d9a2-e1d2-4561-a238-01f8fbb0c0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.496614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085d9a2-e1d2-4561-a238-01f8fbb0c0a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.497252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085d9a2-e1d2-4561-a238-01f8fbb0c0a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.499295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085d9a2-e1d2-4561-a238-01f8fbb0c0a5 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.527841 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1037/2081] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:50 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:50.529078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b358c-acf9-4885-ab75-2bdca4bb3fca admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.540903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b358c-acf9-4885-ab75-2bdca4bb3fca admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.545483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06fb81-6f18-426d-85eb-0c0be514e48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.546097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06fb81-6f18-426d-85eb-0c0be514e48c admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.546375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06fb81-6f18-426d-85eb-0c0be514e48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.546646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06fb81-6f18-426d-85eb-0c0be514e48c admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.547311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06fb81-6f18-426d-85eb-0c0be514e48c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.548944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06fb81-6f18-426d-85eb-0c0be514e48c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.555567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:50.561357 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1045/2082] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:50.565465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085d9a2-e1d2-4561-a238-01f8fbb0c0a5 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.576438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085d9a2-e1d2-4561-a238-01f8fbb0c0a5 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.582553 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab47a6-1687-4c40-a166-2fb85db35969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.583212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab47a6-1687-4c40-a166-2fb85db35969 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.583492 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab47a6-1687-4c40-a166-2fb85db35969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.583767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab47a6-1687-4c40-a166-2fb85db35969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.584497 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab47a6-1687-4c40-a166-2fb85db35969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.586166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab47a6-1687-4c40-a166-2fb85db35969 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.601157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1038/2083] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:50.605631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06fb81-6f18-426d-85eb-0c0be514e48c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.617416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06fb81-6f18-426d-85eb-0c0be514e48c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.622363 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8c2e7-0107-4540-b7a2-12b1d84e3a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.623041 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8c2e7-0107-4540-b7a2-12b1d84e3a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.623383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8c2e7-0107-4540-b7a2-12b1d84e3a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.623684 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8c2e7-0107-4540-b7a2-12b1d84e3a6b admin admin] PATH_INFO: `/v3/projects/0710a9b369ce45a7b9496b7d5f20ff6c/users/3c1b52134f074abf84b62f51f43ecb38/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.643720 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8c2e7-0107-4540-b7a2-12b1d84e3a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0710a9b369ce45a7b9496b7d5f20ff6c, user_id=3c1b52134f074abf84b62f51f43ecb38, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.647278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8c2e7-0107-4540-b7a2-12b1d84e3a6b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.670069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1046/2084] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:50 2022] PUT /identity/v3/projects/0710a9b369ce45a7b9496b7d5f20ff6c/users/3c1b52134f074abf84b62f51f43ecb38/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:50.682238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f4d3a-c424-46db-8a5b-3a83b72ec15f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.682797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f4d3a-c424-46db-8a5b-3a83b72ec15f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.682797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f4d3a-c424-46db-8a5b-3a83b72ec15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.719455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-4c0f4d3a-c424-46db-8a5b-3a83b72ec15f None None] MFA Rules not processed for user `3c1b52134f074abf84b62f51f43ecb38`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:50.720756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4c0f4d3a-c424-46db-8a5b-3a83b72ec15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:50.795368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1039/2085] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:50.807473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8c2e7-0107-4540-b7a2-12b1d84e3a6b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.809293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6de8c2e7-0107-4540-b7a2-12b1d84e3a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:50.876825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8c2e7-0107-4540-b7a2-12b1d84e3a6b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.882010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa5263-1b41-4a08-a322-c0cc16cfe2ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.882703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa5263-1b41-4a08-a322-c0cc16cfe2ed service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.883022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa5263-1b41-4a08-a322-c0cc16cfe2ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.883293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa5263-1b41-4a08-a322-c0cc16cfe2ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.894393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa5263-1b41-4a08-a322-c0cc16cfe2ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.895938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa5263-1b41-4a08-a322-c0cc16cfe2ed service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.906615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1047/2086] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:50.933639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f4d3a-c424-46db-8a5b-3a83b72ec15f None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.965628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:51.004827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f4d3a-c424-46db-8a5b-3a83b72ec15f None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.010016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.011169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.011513 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.011853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] PATH_INFO: `/v3/users/3c1b52134f074abf84b62f51f43ecb38` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.013260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c1b52134f074abf84b62f51f43ecb38)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.015403 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.065987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c1b52134f074abf84b62f51f43ecb38'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.074756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c1b52134f074abf84b62f51f43ecb38'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.075619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c1b52134f074abf84b62f51f43ecb38'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.076253 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c1b52134f074abf84b62f51f43ecb38'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.090620 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c1b52134f074abf84b62f51f43ecb38'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.103176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1040/2087] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:50 2022] DELETE /identity/v3/users/3c1b52134f074abf84b62f51f43ecb38 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:51.107680 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cec2d3-0f6e-49b5-a640-07e079860153 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.107968 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cec2d3-0f6e-49b5-a640-07e079860153 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.108270 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cec2d3-0f6e-49b5-a640-07e079860153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.128261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-c0cec2d3-0f6e-49b5-a640-07e079860153 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:51.130069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c0cec2d3-0f6e-49b5-a640-07e079860153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:51.192486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1048/2088] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:51.202811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.204440 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:51.266162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b12e0-72ba-4f3f-9fe5-e853cde95a17 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.270866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d7e5a-e8e9-4e1d-9614-3842c4e3f3de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.271480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d7e5a-e8e9-4e1d-9614-3842c4e3f3de service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.271760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d7e5a-e8e9-4e1d-9614-3842c4e3f3de service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.272057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d7e5a-e8e9-4e1d-9614-3842c4e3f3de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.281775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d7e5a-e8e9-4e1d-9614-3842c4e3f3de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.282992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d7e5a-e8e9-4e1d-9614-3842c4e3f3de service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.293189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1041/2089] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:51.624779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cec2d3-0f6e-49b5-a640-07e079860153 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.626460 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c0cec2d3-0f6e-49b5-a640-07e079860153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:51.639708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cec2d3-0f6e-49b5-a640-07e079860153 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.644891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22466e-d0ec-4974-bc15-76fcee25abe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.645534 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22466e-d0ec-4974-bc15-76fcee25abe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.645813 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22466e-d0ec-4974-bc15-76fcee25abe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.646081 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22466e-d0ec-4974-bc15-76fcee25abe6 admin admin] PATH_INFO: `/v3/projects/0710a9b369ce45a7b9496b7d5f20ff6c` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.647216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22466e-d0ec-4974-bc15-76fcee25abe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0710a9b369ce45a7b9496b7d5f20ff6c)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.649205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22466e-d0ec-4974-bc15-76fcee25abe6 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.702257 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7a22466e-d0ec-4974-bc15-76fcee25abe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0710a9b369ce45a7b9496b7d5f20ff6c', 'request_id': 'req-7a22466e-d0ec-4974-bc15-76fcee25abe6'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.710927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7a22466e-d0ec-4974-bc15-76fcee25abe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0710a9b369ce45a7b9496b7d5f20ff6c', 'request_id': 'req-7a22466e-d0ec-4974-bc15-76fcee25abe6'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.719447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7a22466e-d0ec-4974-bc15-76fcee25abe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0710a9b369ce45a7b9496b7d5f20ff6c', 'request_id': 'req-7a22466e-d0ec-4974-bc15-76fcee25abe6'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.732425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7a22466e-d0ec-4974-bc15-76fcee25abe6 admin admin] The token cache is being invalidate because project 0710a9b369ce45a7b9496b7d5f20ff6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:51.732776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7a22466e-d0ec-4974-bc15-76fcee25abe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.733516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7a22466e-d0ec-4974-bc15-76fcee25abe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.735352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1049/2090] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:51 2022] DELETE /identity/v3/projects/0710a9b369ce45a7b9496b7d5f20ff6c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:51.741034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d559f1-7c51-4ba4-b392-c72ddf24e788 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.741811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d559f1-7c51-4ba4-b392-c72ddf24e788 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.741988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d559f1-7c51-4ba4-b392-c72ddf24e788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.765643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-81d559f1-7c51-4ba4-b392-c72ddf24e788 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:51.767608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-81d559f1-7c51-4ba4-b392-c72ddf24e788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:51.843734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:51.884433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1042/2091] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:51.888716 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c353b-d663-4022-8b5e-286ba237c817 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.889140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c353b-d663-4022-8b5e-286ba237c817 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.889565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c353b-d663-4022-8b5e-286ba237c817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.899539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:51.922399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-4c7c353b-d663-4022-8b5e-286ba237c817 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:51.923931 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4c7c353b-d663-4022-8b5e-286ba237c817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:51.933386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1050/2092] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:51.937992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d559f1-7c51-4ba4-b392-c72ddf24e788 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.951329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d559f1-7c51-4ba4-b392-c72ddf24e788 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.958230 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888b7e-1c6b-47e8-a4f8-97ad1e763590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.958860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888b7e-1c6b-47e8-a4f8-97ad1e763590 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.959141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888b7e-1c6b-47e8-a4f8-97ad1e763590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.959412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888b7e-1c6b-47e8-a4f8-97ad1e763590 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.960102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39888b7e-1c6b-47e8-a4f8-97ad1e763590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:51.960407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39888b7e-1c6b-47e8-a4f8-97ad1e763590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.962053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39888b7e-1c6b-47e8-a4f8-97ad1e763590 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.992117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1043/2093] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:51.994371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c353b-d663-4022-8b5e-286ba237c817 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:52.012869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c353b-d663-4022-8b5e-286ba237c817 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:52.023328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924928a-bfe0-440f-bf3b-eec7e0b7cfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:52.024520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924928a-bfe0-440f-bf3b-eec7e0b7cfcc admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:52.025031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924928a-bfe0-440f-bf3b-eec7e0b7cfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:52.025424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924928a-bfe0-440f-bf3b-eec7e0b7cfcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:52.026250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924928a-bfe0-440f-bf3b-eec7e0b7cfcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:52.029335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924928a-bfe0-440f-bf3b-eec7e0b7cfcc admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:52.064559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1051/2094] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:52.071659 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888b7e-1c6b-47e8-a4f8-97ad1e763590 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:52.082542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888b7e-1c6b-47e8-a4f8-97ad1e763590 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:52.153097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4f7c7-7395-470e-ba6f-d61e4e942985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:52.153704 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4f7c7-7395-470e-ba6f-d61e4e942985 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:52.153975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4f7c7-7395-470e-ba6f-d61e4e942985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:52.154240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4f7c7-7395-470e-ba6f-d61e4e942985 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:52.154960 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4f7c7-7395-470e-ba6f-d61e4e942985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:52.156953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4f7c7-7395-470e-ba6f-d61e4e942985 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:52.186450 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1044/2095] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:52 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:52.188069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924928a-bfe0-440f-bf3b-eec7e0b7cfcc admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:52.200107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924928a-bfe0-440f-bf3b-eec7e0b7cfcc admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:52.206526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264126e-ff5a-43d9-a4dd-81c0686363c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:52.207017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264126e-ff5a-43d9-a4dd-81c0686363c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:52.207315 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264126e-ff5a-43d9-a4dd-81c0686363c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:52.207663 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264126e-ff5a-43d9-a4dd-81c0686363c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:52.208622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264126e-ff5a-43d9-a4dd-81c0686363c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:52.210470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264126e-ff5a-43d9-a4dd-81c0686363c8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:52.224654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1052/2096] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:52.228765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4f7c7-7395-470e-ba6f-d61e4e942985 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:52.239442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4f7c7-7395-470e-ba6f-d61e4e942985 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:52.244825 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa88cbf-725e-4664-8787-89ef9e929577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:52.245432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa88cbf-725e-4664-8787-89ef9e929577 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:52.245707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa88cbf-725e-4664-8787-89ef9e929577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:52.245976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa88cbf-725e-4664-8787-89ef9e929577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:52.246625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa88cbf-725e-4664-8787-89ef9e929577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:52.248240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa88cbf-725e-4664-8787-89ef9e929577 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:52.263332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1045/2097] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:52.264471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264126e-ff5a-43d9-a4dd-81c0686363c8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:52.276429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264126e-ff5a-43d9-a4dd-81c0686363c8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:52.281158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966b402-79fa-453d-ada2-5887c37264d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:52.281807 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966b402-79fa-453d-ada2-5887c37264d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:52.282126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966b402-79fa-453d-ada2-5887c37264d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:52.282417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966b402-79fa-453d-ada2-5887c37264d0 admin admin] PATH_INFO: `/v3/projects/aaac5ef7406349b298a290f8a1014c63/users/737b2c4d49344733a78c67dfcc1bb39d/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:52.297562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966b402-79fa-453d-ada2-5887c37264d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaac5ef7406349b298a290f8a1014c63, user_id=737b2c4d49344733a78c67dfcc1bb39d, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:52.301019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966b402-79fa-453d-ada2-5887c37264d0 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:52.313255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1053/2098] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:52 2022] PUT /identity/v3/projects/aaac5ef7406349b298a290f8a1014c63/users/737b2c4d49344733a78c67dfcc1bb39d/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:52.324095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ae7c3-0f42-489e-9bf9-04d86789c53a None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:52.324448 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ae7c3-0f42-489e-9bf9-04d86789c53a None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:52.324784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ae7c3-0f42-489e-9bf9-04d86789c53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:52.354638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c57ae7c3-0f42-489e-9bf9-04d86789c53a None None] MFA Rules not processed for user `737b2c4d49344733a78c67dfcc1bb39d`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:52.355905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c57ae7c3-0f42-489e-9bf9-04d86789c53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:52.422322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1046/2099] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:52.436773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966b402-79fa-453d-ada2-5887c37264d0 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:52.438643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0966b402-79fa-453d-ada2-5887c37264d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:52.507407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966b402-79fa-453d-ada2-5887c37264d0 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:52.516146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd1a06-6181-4f3d-82a9-c42d5c3bdf00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:52.516695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd1a06-6181-4f3d-82a9-c42d5c3bdf00 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:52.517043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd1a06-6181-4f3d-82a9-c42d5c3bdf00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:52.517300 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd1a06-6181-4f3d-82a9-c42d5c3bdf00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:52.540135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd1a06-6181-4f3d-82a9-c42d5c3bdf00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:52.540135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd1a06-6181-4f3d-82a9-c42d5c3bdf00 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:52.541652 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1054/2100] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:59.116368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ae7c3-0f42-489e-9bf9-04d86789c53a None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:59.117951 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c57ae7c3-0f42-489e-9bf9-04d86789c53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:59.170485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:59.213129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ae7c3-0f42-489e-9bf9-04d86789c53a None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:59.222827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:59.223623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:59.224006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:59.224609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] PATH_INFO: `/v3/users/e7042e7f6a8c4b188440aa234c78456c` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:59.226192 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7042e7f6a8c4b188440aa234c78456c)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:59.228417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:59.329722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7042e7f6a8c4b188440aa234c78456c'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:59.340836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7042e7f6a8c4b188440aa234c78456c'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:59.341716 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7042e7f6a8c4b188440aa234c78456c'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:59.342550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7042e7f6a8c4b188440aa234c78456c'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:59.366713 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7042e7f6a8c4b188440aa234c78456c'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:59.396262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1047/2101] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:59 2022] DELETE /identity/v3/users/e7042e7f6a8c4b188440aa234c78456c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:59.396371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea58e48-4ad5-437e-823c-7978a3851f28 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:59.396371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea58e48-4ad5-437e-823c-7978a3851f28 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:59.396371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea58e48-4ad5-437e-823c-7978a3851f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:59.411887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6ea58e48-4ad5-437e-823c-7978a3851f28 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:59.413253 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6ea58e48-4ad5-437e-823c-7978a3851f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:59.491838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1055/2102] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:59.520980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:59.522825 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:59.625808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad11017-6f15-4728-b13b-55eaab540aeb admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:59.643447 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e9b23-77ee-4d88-bd5d-207c83a1682d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:59.644432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e9b23-77ee-4d88-bd5d-207c83a1682d service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:59.644711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e9b23-77ee-4d88-bd5d-207c83a1682d service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:59.644985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e9b23-77ee-4d88-bd5d-207c83a1682d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:59.665038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e9b23-77ee-4d88-bd5d-207c83a1682d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:59.666481 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e9b23-77ee-4d88-bd5d-207c83a1682d service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:59.703151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1048/2103] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:00.654824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea58e48-4ad5-437e-823c-7978a3851f28 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:00.659009 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6ea58e48-4ad5-437e-823c-7978a3851f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:00.675183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea58e48-4ad5-437e-823c-7978a3851f28 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:00.682778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:00.683598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:00.684048 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:00.684396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] PATH_INFO: `/v3/projects/583d6d4f47e84344b20592e2e2001705` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:00.692459 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=583d6d4f47e84344b20592e2e2001705)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:00.694572 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:00.781444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:00.801314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583d6d4f47e84344b20592e2e2001705', 'request_id': 'req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:00.808567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583d6d4f47e84344b20592e2e2001705', 'request_id': 'req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:00.818921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583d6d4f47e84344b20592e2e2001705', 'request_id': 'req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:00.833955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] The token cache is being invalidate because project 583d6d4f47e84344b20592e2e2001705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:00.834565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:00.835488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:00.841362 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1056/2104] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:00 2022] DELETE /identity/v3/projects/583d6d4f47e84344b20592e2e2001705 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:00.842635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e9b23-77ee-4d88-bd5d-207c83a1682d service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:00.868173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e9b23-77ee-4d88-bd5d-207c83a1682d service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:00.968432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a54169-a9c1-4c43-bfcb-bc482edfb43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:00.969108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a54169-a9c1-4c43-bfcb-bc482edfb43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:00.969875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a54169-a9c1-4c43-bfcb-bc482edfb43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:00.970207 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a54169-a9c1-4c43-bfcb-bc482edfb43a admin admin] PATH_INFO: `/v3/users/6949d404a825497faa77823e5eed12b6` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:00.972032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a54169-a9c1-4c43-bfcb-bc482edfb43a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6949d404a825497faa77823e5eed12b6)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:00.974025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a54169-a9c1-4c43-bfcb-bc482edfb43a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:00.978933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:00.995632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:00.997478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:01.014322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537cf8e-8959-4d62-a2a1-6019e4e8a7cb admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:01.023333 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:01.024161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:01.024535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:01.024839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] PATH_INFO: `/v3/users/77791e9ca3f74af08b74ddf175bfca6a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:01.026140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77791e9ca3f74af08b74ddf175bfca6a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:01.028455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:01.066171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f5a54169-a9c1-4c43-bfcb-bc482edfb43a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6949d404a825497faa77823e5eed12b6'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.081422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f5a54169-a9c1-4c43-bfcb-bc482edfb43a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6949d404a825497faa77823e5eed12b6'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.083069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f5a54169-a9c1-4c43-bfcb-bc482edfb43a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6949d404a825497faa77823e5eed12b6'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.083914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f5a54169-a9c1-4c43-bfcb-bc482edfb43a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6949d404a825497faa77823e5eed12b6'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.103571 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f5a54169-a9c1-4c43-bfcb-bc482edfb43a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6949d404a825497faa77823e5eed12b6'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.123204 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77791e9ca3f74af08b74ddf175bfca6a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.126466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1049/2105] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:00 2022] DELETE /identity/v3/users/6949d404a825497faa77823e5eed12b6 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:01.150381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77791e9ca3f74af08b74ddf175bfca6a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.164950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77791e9ca3f74af08b74ddf175bfca6a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.177981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77791e9ca3f74af08b74ddf175bfca6a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.178741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77791e9ca3f74af08b74ddf175bfca6a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.198164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1057/2106] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:00 2022] DELETE /identity/v3/users/77791e9ca3f74af08b74ddf175bfca6a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:01.200472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3752c3-778e-40ef-b632-5406c4edf1c6 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:01.202016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3752c3-778e-40ef-b632-5406c4edf1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:01.202690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3752c3-778e-40ef-b632-5406c4edf1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:01.253795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-4b3752c3-778e-40ef-b632-5406c4edf1c6 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:01.255420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4b3752c3-778e-40ef-b632-5406c4edf1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:01.347578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1050/2107] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:01.363791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:01.365267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:01.441315 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48152a0-5bb6-4dda-9cf1-4cdf575aede7 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:01.446120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf580c-e0a3-45d5-a1b4-5ac50e93f5bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:01.446854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf580c-e0a3-45d5-a1b4-5ac50e93f5bb service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:01.447140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf580c-e0a3-45d5-a1b4-5ac50e93f5bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:01.448157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf580c-e0a3-45d5-a1b4-5ac50e93f5bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:01.459433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf580c-e0a3-45d5-a1b4-5ac50e93f5bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:01.461110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf580c-e0a3-45d5-a1b4-5ac50e93f5bb service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:01.477757 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1058/2108] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:01.649765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3752c3-778e-40ef-b632-5406c4edf1c6 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:01.653979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4b3752c3-778e-40ef-b632-5406c4edf1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:01.695726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3752c3-778e-40ef-b632-5406c4edf1c6 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:01.715279 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:01.715973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:01.716285 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:01.716663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] PATH_INFO: `/v3/projects/0fd0e8bbce8d4f3fb29cad967f9c55a8` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:01.717878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd0e8bbce8d4f3fb29cad967f9c55a8)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:01.719878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:01.800259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd0e8bbce8d4f3fb29cad967f9c55a8', 'request_id': 'req-793be578-31f4-4019-b80b-81b99fab03ad'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.812281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd0e8bbce8d4f3fb29cad967f9c55a8', 'request_id': 'req-793be578-31f4-4019-b80b-81b99fab03ad'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.820339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd0e8bbce8d4f3fb29cad967f9c55a8', 'request_id': 'req-793be578-31f4-4019-b80b-81b99fab03ad'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.835961 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] The token cache is being invalidate because project 0fd0e8bbce8d4f3fb29cad967f9c55a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:01.836299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.836929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:01.838827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1051/2109] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:01 2022] DELETE /identity/v3/projects/0fd0e8bbce8d4f3fb29cad967f9c55a8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:01.843746 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4f762-f554-42fb-ade7-54e36af156b3 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:01.844296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4f762-f554-42fb-ade7-54e36af156b3 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:01.844468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4f762-f554-42fb-ade7-54e36af156b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:01.867148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-22d4f762-f554-42fb-ade7-54e36af156b3 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:01.868628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-22d4f762-f554-42fb-ade7-54e36af156b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:01.919898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:01.938545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:01.941570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:01.951690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:01.984725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1059/2110] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:01.994407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469bd4e-d5c0-4128-bd17-282b10e7f96d None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:01.994516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469bd4e-d5c0-4128-bd17-282b10e7f96d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:01.994998 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469bd4e-d5c0-4128-bd17-282b10e7f96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:02.028881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-2469bd4e-d5c0-4128-bd17-282b10e7f96d None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:02.030149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2469bd4e-d5c0-4128-bd17-282b10e7f96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:02.048126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1060/2111] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:02.053029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469bd4e-d5c0-4128-bd17-282b10e7f96d None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:02.081683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469bd4e-d5c0-4128-bd17-282b10e7f96d None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:02.086680 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3825631-d49b-4fc8-b071-358dce3ad393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:02.087383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3825631-d49b-4fc8-b071-358dce3ad393 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:02.087733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3825631-d49b-4fc8-b071-358dce3ad393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:02.088093 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3825631-d49b-4fc8-b071-358dce3ad393 admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:02.089676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3825631-d49b-4fc8-b071-358dce3ad393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:02.090062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3825631-d49b-4fc8-b071-358dce3ad393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:02.092032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3825631-d49b-4fc8-b071-358dce3ad393 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:02.112769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1061/2112] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:02.116224 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793be578-31f4-4019-b80b-81b99fab03ad admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:02.117354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3825631-d49b-4fc8-b071-358dce3ad393 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:02.133616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:02.135009 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:02.135470 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:02.135895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] PATH_INFO: `/v3/projects/ef940fad71114e0a80124e1829353bd5` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:02.137783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef940fad71114e0a80124e1829353bd5)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:02.140199 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:02.143549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3825631-d49b-4fc8-b071-358dce3ad393 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:02.150298 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa44ecd-3914-45be-8639-937a96f34461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:02.150936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa44ecd-3914-45be-8639-937a96f34461 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:02.151280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa44ecd-3914-45be-8639-937a96f34461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:02.151558 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa44ecd-3914-45be-8639-937a96f34461 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:02.152250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa44ecd-3914-45be-8639-937a96f34461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:02.154230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa44ecd-3914-45be-8639-937a96f34461 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:02.204651 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1062/2113] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:02 2022] POST /identity/v3/projects => generated 367 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:02.219151 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa44ecd-3914-45be-8639-937a96f34461 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:02.235011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef940fad71114e0a80124e1829353bd5', 'request_id': 'req-d66331e7-9db2-43e6-932c-8249195a33ef'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:02.243587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef940fad71114e0a80124e1829353bd5', 'request_id': 'req-d66331e7-9db2-43e6-932c-8249195a33ef'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:02.253366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef940fad71114e0a80124e1829353bd5', 'request_id': 'req-d66331e7-9db2-43e6-932c-8249195a33ef'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:02.274614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] The token cache is being invalidate because project ef940fad71114e0a80124e1829353bd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:02.275211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:02.276761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:02.279652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1052/2114] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:01 2022] DELETE /identity/v3/projects/ef940fad71114e0a80124e1829353bd5 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:02.284810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:02.285932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:02.359569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa44ecd-3914-45be-8639-937a96f34461 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:02.467292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561233b1-b046-44fa-a62c-c6f1dfab20c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:02.468039 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561233b1-b046-44fa-a62c-c6f1dfab20c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:02.468325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561233b1-b046-44fa-a62c-c6f1dfab20c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:02.468722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561233b1-b046-44fa-a62c-c6f1dfab20c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:02.469481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561233b1-b046-44fa-a62c-c6f1dfab20c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:02.472102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561233b1-b046-44fa-a62c-c6f1dfab20c3 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:02.477392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66331e7-9db2-43e6-932c-8249195a33ef admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:02.496206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8a0ce-7de5-4622-98be-cd53c5918813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:02.497762 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8a0ce-7de5-4622-98be-cd53c5918813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:02.498538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8a0ce-7de5-4622-98be-cd53c5918813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:02.499290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8a0ce-7de5-4622-98be-cd53c5918813 admin admin] PATH_INFO: `/v3/users/c6a2c2455ac44bedb4bbf62bb11639bc` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:02.502611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e8a0ce-7de5-4622-98be-cd53c5918813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a2c2455ac44bedb4bbf62bb11639bc)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:02.507563 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e8a0ce-7de5-4622-98be-cd53c5918813 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:02.511689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1063/2115] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:02 2022] POST /identity/v3/users => generated 284 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:02.516655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561233b1-b046-44fa-a62c-c6f1dfab20c3 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:02.519279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-561233b1-b046-44fa-a62c-c6f1dfab20c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:02.546072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561233b1-b046-44fa-a62c-c6f1dfab20c3 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:02.554467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a6d3a-aa31-4ce1-9f71-f55dace47d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:02.555165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a6d3a-aa31-4ce1-9f71-f55dace47d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:02.555496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a6d3a-aa31-4ce1-9f71-f55dace47d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:02.555775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a6d3a-aa31-4ce1-9f71-f55dace47d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:02.556518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a6d3a-aa31-4ce1-9f71-f55dace47d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:02.560503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a6d3a-aa31-4ce1-9f71-f55dace47d41 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:02.591307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1064/2116] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:02.595745 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:02.596695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a6d3a-aa31-4ce1-9f71-f55dace47d41 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:02.613009 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-37e8a0ce-7de5-4622-98be-cd53c5918813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a2c2455ac44bedb4bbf62bb11639bc'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:02.624344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a6d3a-aa31-4ce1-9f71-f55dace47d41 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:02.625780 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-37e8a0ce-7de5-4622-98be-cd53c5918813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a2c2455ac44bedb4bbf62bb11639bc'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:02.627162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-37e8a0ce-7de5-4622-98be-cd53c5918813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a2c2455ac44bedb4bbf62bb11639bc'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:02.627905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-37e8a0ce-7de5-4622-98be-cd53c5918813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a2c2455ac44bedb4bbf62bb11639bc'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:02.632679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2febb6-7766-4402-9e8e-73abb93b8a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:02.633319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2febb6-7766-4402-9e8e-73abb93b8a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:02.633597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2febb6-7766-4402-9e8e-73abb93b8a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:02.637733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2febb6-7766-4402-9e8e-73abb93b8a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:02.639344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2febb6-7766-4402-9e8e-73abb93b8a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:02.641652 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2febb6-7766-4402-9e8e-73abb93b8a41 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:02.658723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-37e8a0ce-7de5-4622-98be-cd53c5918813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a2c2455ac44bedb4bbf62bb11639bc'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:02.672313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1065/2117] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:02.677369 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2febb6-7766-4402-9e8e-73abb93b8a41 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:02.681019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1053/2118] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:02 2022] DELETE /identity/v3/users/c6a2c2455ac44bedb4bbf62bb11639bc => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:02.683376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7b2febb6-7766-4402-9e8e-73abb93b8a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:02.705877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2febb6-7766-4402-9e8e-73abb93b8a41 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:02.726029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:02.798715 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1290c-2155-40ca-be49-e31490b2ffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:02.800025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1290c-2155-40ca-be49-e31490b2ffc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:02.800025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1290c-2155-40ca-be49-e31490b2ffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:02.800460 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1290c-2155-40ca-be49-e31490b2ffc8 admin admin] PATH_INFO: `/v3/projects/1cf4f0c64dad45a9a46f00590f2c24c5/users/aa0b29fa73cf48a1bec3253f89f01377/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:02.820769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1290c-2155-40ca-be49-e31490b2ffc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf4f0c64dad45a9a46f00590f2c24c5, user_id=aa0b29fa73cf48a1bec3253f89f01377, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:02.821561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1290c-2155-40ca-be49-e31490b2ffc8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:02.841494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1066/2119] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:02 2022] PUT /identity/v3/projects/1cf4f0c64dad45a9a46f00590f2c24c5/users/aa0b29fa73cf48a1bec3253f89f01377/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:02.853244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c06966-288c-486e-9c66-a7683e36df1c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:02.853587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c06966-288c-486e-9c66-a7683e36df1c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:02.853790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c06966-288c-486e-9c66-a7683e36df1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:02.889366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-b0c06966-288c-486e-9c66-a7683e36df1c None None] MFA Rules not processed for user `aa0b29fa73cf48a1bec3253f89f01377`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:02.890819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b0c06966-288c-486e-9c66-a7683e36df1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:02.982907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1054/2120] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:02.984804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1290c-2155-40ca-be49-e31490b2ffc8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:03.063510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c06966-288c-486e-9c66-a7683e36df1c None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:03.066061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b0c06966-288c-486e-9c66-a7683e36df1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:03.121604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1290c-2155-40ca-be49-e31490b2ffc8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:03.131535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f327d-1b74-424e-9cd1-08ff947387a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:03.132206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f327d-1b74-424e-9cd1-08ff947387a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:03.132480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f327d-1b74-424e-9cd1-08ff947387a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:03.132748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f327d-1b74-424e-9cd1-08ff947387a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:03.133404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f327d-1b74-424e-9cd1-08ff947387a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:03.135881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f327d-1b74-424e-9cd1-08ff947387a3 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:03.177928 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1067/2121] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:02 2022] POST /identity/v3/projects => generated 367 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:03.182947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f327d-1b74-424e-9cd1-08ff947387a3 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:03.192456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c06966-288c-486e-9c66-a7683e36df1c None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:03.294410 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:03.322102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e56f52-e4f6-468b-89f0-264eb40c7544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:03.322929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e56f52-e4f6-468b-89f0-264eb40c7544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:03.323218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e56f52-e4f6-468b-89f0-264eb40c7544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:03.323487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e56f52-e4f6-468b-89f0-264eb40c7544 admin admin] PATH_INFO: `/v3/projects/7e934b7ced8546fdb8d3606ce3208e8f` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:03.324815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e56f52-e4f6-468b-89f0-264eb40c7544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e934b7ced8546fdb8d3606ce3208e8f)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:03.328255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e56f52-e4f6-468b-89f0-264eb40c7544 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:03.331426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f327d-1b74-424e-9cd1-08ff947387a3 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:03.338528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c85309-0cda-40c6-8e74-f1106586ced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:03.339681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c85309-0cda-40c6-8e74-f1106586ced4 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:03.344552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c85309-0cda-40c6-8e74-f1106586ced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:03.345767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c85309-0cda-40c6-8e74-f1106586ced4 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:03.346781 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c85309-0cda-40c6-8e74-f1106586ced4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:03.349679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c85309-0cda-40c6-8e74-f1106586ced4 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:03.383409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1068/2122] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:03 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:03.390063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c85309-0cda-40c6-8e74-f1106586ced4 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:03.404213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c85309-0cda-40c6-8e74-f1106586ced4 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:03.417222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94adeab-0b80-4591-88e4-ce27021ee310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:03.418495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94adeab-0b80-4591-88e4-ce27021ee310 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:03.419080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94adeab-0b80-4591-88e4-ce27021ee310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:03.420013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94adeab-0b80-4591-88e4-ce27021ee310 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:03.421089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94adeab-0b80-4591-88e4-ce27021ee310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:03.425582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94adeab-0b80-4591-88e4-ce27021ee310 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:03.436264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:03.461131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1069/2123] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:03.485212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94adeab-0b80-4591-88e4-ce27021ee310 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:03.490738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-95e56f52-e4f6-468b-89f0-264eb40c7544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e934b7ced8546fdb8d3606ce3208e8f', 'request_id': 'req-95e56f52-e4f6-468b-89f0-264eb40c7544'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:03.516439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94adeab-0b80-4591-88e4-ce27021ee310 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:03.520937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-95e56f52-e4f6-468b-89f0-264eb40c7544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e934b7ced8546fdb8d3606ce3208e8f', 'request_id': 'req-95e56f52-e4f6-468b-89f0-264eb40c7544'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:03.588717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-95e56f52-e4f6-468b-89f0-264eb40c7544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e934b7ced8546fdb8d3606ce3208e8f', 'request_id': 'req-95e56f52-e4f6-468b-89f0-264eb40c7544'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:03.675526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-95e56f52-e4f6-468b-89f0-264eb40c7544 admin admin] The token cache is being invalidate because project 7e934b7ced8546fdb8d3606ce3208e8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:03.676506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-95e56f52-e4f6-468b-89f0-264eb40c7544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:03.679325 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-95e56f52-e4f6-468b-89f0-264eb40c7544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:03.683450 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1055/2124] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:03 2022] DELETE /identity/v3/projects/7e934b7ced8546fdb8d3606ce3208e8f => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:03.696518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac865d-07f4-4374-bf4f-b7f6aab8b989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:03.697275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac865d-07f4-4374-bf4f-b7f6aab8b989 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:03.697601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac865d-07f4-4374-bf4f-b7f6aab8b989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:03.698146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac865d-07f4-4374-bf4f-b7f6aab8b989 admin admin] PATH_INFO: `/v3/projects/220f3c73702f457aa101e3df678c4c8f/users/060c483874694eb6867d461f3e0dbe81/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:03.700296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ea73e-27e7-4afc-9326-d13fafa2d582 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:03.701292 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ea73e-27e7-4afc-9326-d13fafa2d582 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:03.704285 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ea73e-27e7-4afc-9326-d13fafa2d582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:03.725853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac865d-07f4-4374-bf4f-b7f6aab8b989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=220f3c73702f457aa101e3df678c4c8f, user_id=060c483874694eb6867d461f3e0dbe81, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:03.733825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac865d-07f4-4374-bf4f-b7f6aab8b989 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:03.768737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1070/2125] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:03 2022] PUT /identity/v3/projects/220f3c73702f457aa101e3df678c4c8f/users/060c483874694eb6867d461f3e0dbe81/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:03.770789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac865d-07f4-4374-bf4f-b7f6aab8b989 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:03.774896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f3ac865d-07f4-4374-bf4f-b7f6aab8b989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:03.777180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-6e9ea73e-27e7-4afc-9326-d13fafa2d582 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:03.778453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6e9ea73e-27e7-4afc-9326-d13fafa2d582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:03.930742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1056/2126] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:03.932169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c855125-0e1d-46f2-a8c6-483eaf07a8bd None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:03.932456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c855125-0e1d-46f2-a8c6-483eaf07a8bd None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:03.932730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c855125-0e1d-46f2-a8c6-483eaf07a8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:03.983461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac865d-07f4-4374-bf4f-b7f6aab8b989 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:03.994963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-7c855125-0e1d-46f2-a8c6-483eaf07a8bd None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:03.997043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4e5e6-b583-4526-9f8c-0abc561622f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:03.998121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7c855125-0e1d-46f2-a8c6-483eaf07a8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:04.001837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4e5e6-b583-4526-9f8c-0abc561622f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.004161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4e5e6-b583-4526-9f8c-0abc561622f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.005032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4e5e6-b583-4526-9f8c-0abc561622f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.006420 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4e5e6-b583-4526-9f8c-0abc561622f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.011036 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4e5e6-b583-4526-9f8c-0abc561622f4 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.027776 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1057/2127] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:04.032086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1071/2128] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:04.033832 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c855125-0e1d-46f2-a8c6-483eaf07a8bd None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:04.037093 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4e5e6-b583-4526-9f8c-0abc561622f4 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:04.047257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c855125-0e1d-46f2-a8c6-483eaf07a8bd None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:04.049085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4e5e6-b583-4526-9f8c-0abc561622f4 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:04.053282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d60166-f2f6-4163-93fd-19908e6089ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:04.053911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d60166-f2f6-4163-93fd-19908e6089ca admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.054187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d60166-f2f6-4163-93fd-19908e6089ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.054453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d60166-f2f6-4163-93fd-19908e6089ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.054821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236df1f9-a16a-4939-9c76-79a5a7aa33d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:04.055111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d60166-f2f6-4163-93fd-19908e6089ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:04.056119 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d60166-f2f6-4163-93fd-19908e6089ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.056329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236df1f9-a16a-4939-9c76-79a5a7aa33d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.056329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236df1f9-a16a-4939-9c76-79a5a7aa33d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.056329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236df1f9-a16a-4939-9c76-79a5a7aa33d5 admin admin] PATH_INFO: `/v3/projects/220f3c73702f457aa101e3df678c4c8f/users/060c483874694eb6867d461f3e0dbe81/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.057271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d60166-f2f6-4163-93fd-19908e6089ca admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.058296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236df1f9-a16a-4939-9c76-79a5a7aa33d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=220f3c73702f457aa101e3df678c4c8f, user_id=060c483874694eb6867d461f3e0dbe81, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.061088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236df1f9-a16a-4939-9c76-79a5a7aa33d5 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.075127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1058/2129] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:04.076491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d60166-f2f6-4163-93fd-19908e6089ca admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:04.077744 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1072/2130] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:04 2022] PUT /identity/v3/projects/220f3c73702f457aa101e3df678c4c8f/users/060c483874694eb6867d461f3e0dbe81/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:04.093803 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be51a35-0257-48aa-a5a2-57bdb514eb80 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.094252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be51a35-0257-48aa-a5a2-57bdb514eb80 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.094376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be51a35-0257-48aa-a5a2-57bdb514eb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.104416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d60166-f2f6-4163-93fd-19908e6089ca admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:04.147175 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-3be51a35-0257-48aa-a5a2-57bdb514eb80 None None] MFA Rules not processed for user `060c483874694eb6867d461f3e0dbe81`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:04.148487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3be51a35-0257-48aa-a5a2-57bdb514eb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:04.204023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b6f6c-11c2-4935-a358-f95239ffb7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:04.205379 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b6f6c-11c2-4935-a358-f95239ffb7d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.210193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b6f6c-11c2-4935-a358-f95239ffb7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.210452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b6f6c-11c2-4935-a358-f95239ffb7d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.211249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b6f6c-11c2-4935-a358-f95239ffb7d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.212397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:04.213409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b6f6c-11c2-4935-a358-f95239ffb7d3 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.221549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:04.230533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1073/2131] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:04.246496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be51a35-0257-48aa-a5a2-57bdb514eb80 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:04.248275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3be51a35-0257-48aa-a5a2-57bdb514eb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:04.258107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1059/2132] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:04 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:04.262911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b6f6c-11c2-4935-a358-f95239ffb7d3 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:04.347165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be51a35-0257-48aa-a5a2-57bdb514eb80 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:04.368980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b6f6c-11c2-4935-a358-f95239ffb7d3 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:04.378926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab9907-213b-4374-b7e7-01d45bc46864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:04.379909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab9907-213b-4374-b7e7-01d45bc46864 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.380638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab9907-213b-4374-b7e7-01d45bc46864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.381016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab9907-213b-4374-b7e7-01d45bc46864 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.381701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab9907-213b-4374-b7e7-01d45bc46864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.384488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab9907-213b-4374-b7e7-01d45bc46864 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.412337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10394497-06a7-4e97-8b75-c2fc85ca21c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:04.413530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10394497-06a7-4e97-8b75-c2fc85ca21c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.413955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10394497-06a7-4e97-8b75-c2fc85ca21c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.414522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10394497-06a7-4e97-8b75-c2fc85ca21c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.420647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1060/2133] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:04 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:04.432748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab9907-213b-4374-b7e7-01d45bc46864 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:04.447928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab9907-213b-4374-b7e7-01d45bc46864 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:04.459199 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8418fc-9572-4308-871a-1dd9b5ad86f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:04.459913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8418fc-9572-4308-871a-1dd9b5ad86f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.460218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8418fc-9572-4308-871a-1dd9b5ad86f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.461030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8418fc-9572-4308-871a-1dd9b5ad86f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.461153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8418fc-9572-4308-871a-1dd9b5ad86f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.462837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8418fc-9572-4308-871a-1dd9b5ad86f7 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.480514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1061/2134] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:04.483413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8418fc-9572-4308-871a-1dd9b5ad86f7 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:04.495097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8418fc-9572-4308-871a-1dd9b5ad86f7 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:04.500975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b576e3-21bc-4a2e-a6e5-2f622775c718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:04.502501 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b576e3-21bc-4a2e-a6e5-2f622775c718 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.502695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b576e3-21bc-4a2e-a6e5-2f622775c718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.503236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b576e3-21bc-4a2e-a6e5-2f622775c718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.504688 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b576e3-21bc-4a2e-a6e5-2f622775c718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.506538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b576e3-21bc-4a2e-a6e5-2f622775c718 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.518357 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10394497-06a7-4e97-8b75-c2fc85ca21c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.519879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10394497-06a7-4e97-8b75-c2fc85ca21c7 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.520830 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1062/2135] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:04.526658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b576e3-21bc-4a2e-a6e5-2f622775c718 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:04.533349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1074/2136] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:04.542142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b576e3-21bc-4a2e-a6e5-2f622775c718 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:04.549931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ab2be-e8de-4355-85d9-721b1b03c812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:04.550741 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ab2be-e8de-4355-85d9-721b1b03c812 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.551120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ab2be-e8de-4355-85d9-721b1b03c812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.551455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ab2be-e8de-4355-85d9-721b1b03c812 admin admin] PATH_INFO: `/v3/projects/f43a1c04555843f9a6bce2f83698d467/users/224b92c1aa704326b6a94e9f6051af07/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.573100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ab2be-e8de-4355-85d9-721b1b03c812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43a1c04555843f9a6bce2f83698d467, user_id=224b92c1aa704326b6a94e9f6051af07, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.575980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ab2be-e8de-4355-85d9-721b1b03c812 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.608162 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1063/2137] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:04 2022] PUT /identity/v3/projects/f43a1c04555843f9a6bce2f83698d467/users/224b92c1aa704326b6a94e9f6051af07/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:04.615270 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826f0a8-e2cb-493d-b927-e59da42103eb None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.615565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826f0a8-e2cb-493d-b927-e59da42103eb None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.615809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826f0a8-e2cb-493d-b927-e59da42103eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.671038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ab2be-e8de-4355-85d9-721b1b03c812 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:04.672770 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b826f0a8-e2cb-493d-b927-e59da42103eb None None] MFA Rules not processed for user `224b92c1aa704326b6a94e9f6051af07`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:04.674944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b826f0a8-e2cb-493d-b927-e59da42103eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:04.774436 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:04.807538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ab2be-e8de-4355-85d9-721b1b03c812 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:04.810485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1075/2138] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:04.811546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0f031-3931-49d9-a4c1-e86c8fd0b1de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:04.812363 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0f031-3931-49d9-a4c1-e86c8fd0b1de service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.812640 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0f031-3931-49d9-a4c1-e86c8fd0b1de service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.812905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0f031-3931-49d9-a4c1-e86c8fd0b1de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.815609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c8d0f031-3931-49d9-a4c1-e86c8fd0b1de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:04.817933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826f0a8-e2cb-493d-b927-e59da42103eb None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:04.889259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:04.916306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0f031-3931-49d9-a4c1-e86c8fd0b1de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.916306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0f031-3931-49d9-a4c1-e86c8fd0b1de service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.921164 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1064/2139] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:04.935763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826f0a8-e2cb-493d-b927-e59da42103eb None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:04.943684 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684dfe1c-c89b-4535-971d-7d5f67687535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:04.944731 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dfe1c-c89b-4535-971d-7d5f67687535 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.945183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dfe1c-c89b-4535-971d-7d5f67687535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.945505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dfe1c-c89b-4535-971d-7d5f67687535 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.946561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684dfe1c-c89b-4535-971d-7d5f67687535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.949236 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684dfe1c-c89b-4535-971d-7d5f67687535 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.997097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1076/2140] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:04 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:05.004170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0f031-3931-49d9-a4c1-e86c8fd0b1de service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:05.018749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0f031-3931-49d9-a4c1-e86c8fd0b1de service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:05.120580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880074b7-4670-4de8-9e2e-d5d680947319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:05.120580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880074b7-4670-4de8-9e2e-d5d680947319 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:05.120580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880074b7-4670-4de8-9e2e-d5d680947319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:05.120580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880074b7-4670-4de8-9e2e-d5d680947319 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:05.121259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880074b7-4670-4de8-9e2e-d5d680947319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:05.129836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880074b7-4670-4de8-9e2e-d5d680947319 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:05.191650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684dfe1c-c89b-4535-971d-7d5f67687535 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:05.203354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1065/2141] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:05 2022] POST /identity/v3/users => generated 299 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:05.241839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684dfe1c-c89b-4535-971d-7d5f67687535 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:05.276935 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e56b2-658d-4e62-aaa2-6a940e80c5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:05.277910 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e56b2-658d-4e62-aaa2-6a940e80c5fc admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:05.280936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e56b2-658d-4e62-aaa2-6a940e80c5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:05.281889 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e56b2-658d-4e62-aaa2-6a940e80c5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:05.285092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e56b2-658d-4e62-aaa2-6a940e80c5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:05.289272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e56b2-658d-4e62-aaa2-6a940e80c5fc admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:05.331303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880074b7-4670-4de8-9e2e-d5d680947319 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:05.332372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1077/2142] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:05.356879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880074b7-4670-4de8-9e2e-d5d680947319 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:05.371756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb31a0c-64f0-435e-8b6b-9b6b9e4b73ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:05.372821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb31a0c-64f0-435e-8b6b-9b6b9e4b73ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:05.373298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb31a0c-64f0-435e-8b6b-9b6b9e4b73ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:05.373836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb31a0c-64f0-435e-8b6b-9b6b9e4b73ab admin admin] PATH_INFO: `/v3/projects/7c4e3e64a3284cddbfd3f8134791dcf0/users/478b3b526a6a4311ae99dc65ffe96b9a/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:05.416577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb31a0c-64f0-435e-8b6b-9b6b9e4b73ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c4e3e64a3284cddbfd3f8134791dcf0, user_id=478b3b526a6a4311ae99dc65ffe96b9a, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:05.420830 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb31a0c-64f0-435e-8b6b-9b6b9e4b73ab admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:05.449945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1066/2143] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:05 2022] PUT /identity/v3/projects/7c4e3e64a3284cddbfd3f8134791dcf0/users/478b3b526a6a4311ae99dc65ffe96b9a/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:05.454682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e56b2-658d-4e62-aaa2-6a940e80c5fc admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:05.476324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e56b2-658d-4e62-aaa2-6a940e80c5fc admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:05.609222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f180732-16de-4935-a376-a95fcb8818ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:05.609909 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f180732-16de-4935-a376-a95fcb8818ad admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:05.610194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f180732-16de-4935-a376-a95fcb8818ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:05.610468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f180732-16de-4935-a376-a95fcb8818ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:05.611710 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f180732-16de-4935-a376-a95fcb8818ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:05.614119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f180732-16de-4935-a376-a95fcb8818ad admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:05.628830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1078/2144] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:05.635062 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb31a0c-64f0-435e-8b6b-9b6b9e4b73ab admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:05.649336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb31a0c-64f0-435e-8b6b-9b6b9e4b73ab admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:05.654282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8d217-2c0a-45dc-9e3c-bdf9bb18a9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:05.654986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8d217-2c0a-45dc-9e3c-bdf9bb18a9e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:05.655293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8d217-2c0a-45dc-9e3c-bdf9bb18a9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:05.655591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8d217-2c0a-45dc-9e3c-bdf9bb18a9e2 admin admin] PATH_INFO: `/v3/projects/7c4e3e64a3284cddbfd3f8134791dcf0/users/478b3b526a6a4311ae99dc65ffe96b9a/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:05.657987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8d217-2c0a-45dc-9e3c-bdf9bb18a9e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c4e3e64a3284cddbfd3f8134791dcf0, user_id=478b3b526a6a4311ae99dc65ffe96b9a, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:05.661261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8d217-2c0a-45dc-9e3c-bdf9bb18a9e2 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:05.674774 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1067/2145] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:05 2022] PUT /identity/v3/projects/7c4e3e64a3284cddbfd3f8134791dcf0/users/478b3b526a6a4311ae99dc65ffe96b9a/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:05.687529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d15c1a-42e8-4668-ac06-2c2800a27b16 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:05.687794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d15c1a-42e8-4668-ac06-2c2800a27b16 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:05.687966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d15c1a-42e8-4668-ac06-2c2800a27b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:05.721860 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d1d15c1a-42e8-4668-ac06-2c2800a27b16 None None] MFA Rules not processed for user `478b3b526a6a4311ae99dc65ffe96b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:05.723157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d1d15c1a-42e8-4668-ac06-2c2800a27b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:05.776518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:05.803155 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1079/2146] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:05.816770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8d217-2c0a-45dc-9e3c-bdf9bb18a9e2 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:05.898093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8d217-2c0a-45dc-9e3c-bdf9bb18a9e2 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:05.904302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6503ae-31ec-493b-aaed-4693c5f35ee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:05.905368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6503ae-31ec-493b-aaed-4693c5f35ee5 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:05.905844 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6503ae-31ec-493b-aaed-4693c5f35ee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:05.906304 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6503ae-31ec-493b-aaed-4693c5f35ee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:05.924441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6503ae-31ec-493b-aaed-4693c5f35ee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:05.927197 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6503ae-31ec-493b-aaed-4693c5f35ee5 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:05.942922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1068/2147] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:06.026730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d15c1a-42e8-4668-ac06-2c2800a27b16 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:06.116425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d15c1a-42e8-4668-ac06-2c2800a27b16 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:06.119905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:06.120679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:06.121062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:06.121276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] PATH_INFO: `/v3/users/224b92c1aa704326b6a94e9f6051af07` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:06.122631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] RBAC: Authorizing `identity:delete_user(user_id=224b92c1aa704326b6a94e9f6051af07)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:06.124913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:06.201790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224b92c1aa704326b6a94e9f6051af07'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.231255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224b92c1aa704326b6a94e9f6051af07'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.247037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224b92c1aa704326b6a94e9f6051af07'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.256892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224b92c1aa704326b6a94e9f6051af07'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.257764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224b92c1aa704326b6a94e9f6051af07'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.272894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1080/2148] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:06 2022] DELETE /identity/v3/users/224b92c1aa704326b6a94e9f6051af07 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:06.277479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d0bee-1c43-4e9e-8951-266785f5340c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:06.277942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d0bee-1c43-4e9e-8951-266785f5340c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:06.278026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d0bee-1c43-4e9e-8951-266785f5340c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:06.304669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d49d0bee-1c43-4e9e-8951-266785f5340c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:06.305949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d49d0bee-1c43-4e9e-8951-266785f5340c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:06.457120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:06.539259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1069/2149] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:06.540547 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:06.545569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:06.671599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd365c-0f85-41a1-9d00-ee5c58ae93df admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:06.679718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33573f9c-bd65-49fa-8369-a47b8327a3c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:06.680768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33573f9c-bd65-49fa-8369-a47b8327a3c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:06.682604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33573f9c-bd65-49fa-8369-a47b8327a3c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:06.682604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33573f9c-bd65-49fa-8369-a47b8327a3c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:06.718376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33573f9c-bd65-49fa-8369-a47b8327a3c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:06.724462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33573f9c-bd65-49fa-8369-a47b8327a3c3 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:06.749047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1081/2150] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:07.442998 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d0bee-1c43-4e9e-8951-266785f5340c None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:07.444670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d49d0bee-1c43-4e9e-8951-266785f5340c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:07.458760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d0bee-1c43-4e9e-8951-266785f5340c None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:07.465290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:07.466134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:07.466579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:07.467017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] PATH_INFO: `/v3/projects/f43a1c04555843f9a6bce2f83698d467` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:07.473696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f43a1c04555843f9a6bce2f83698d467)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:07.477430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:07.560611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f43a1c04555843f9a6bce2f83698d467', 'request_id': 'req-6fd81845-8f92-48fa-a746-ea88d97e8e7d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:07.577074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f43a1c04555843f9a6bce2f83698d467', 'request_id': 'req-6fd81845-8f92-48fa-a746-ea88d97e8e7d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:07.586155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f43a1c04555843f9a6bce2f83698d467', 'request_id': 'req-6fd81845-8f92-48fa-a746-ea88d97e8e7d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:07.607310 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] The token cache is being invalidate because project f43a1c04555843f9a6bce2f83698d467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:07.607791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:07.608936 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:07.612013 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1070/2151] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:07 2022] DELETE /identity/v3/projects/f43a1c04555843f9a6bce2f83698d467 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:07.617819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33573f9c-bd65-49fa-8369-a47b8327a3c3 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:07.625490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:07.634560 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33573f9c-bd65-49fa-8369-a47b8327a3c3 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:07.734239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:07.738968 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:07.779517 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac859d29-a324-4b7f-9c9a-00ed9a8b5b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:07.780715 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859d29-a324-4b7f-9c9a-00ed9a8b5b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:07.781216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859d29-a324-4b7f-9c9a-00ed9a8b5b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:07.781718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859d29-a324-4b7f-9c9a-00ed9a8b5b7a admin admin] PATH_INFO: `/v3/users/478b3b526a6a4311ae99dc65ffe96b9a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:07.784467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac859d29-a324-4b7f-9c9a-00ed9a8b5b7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=478b3b526a6a4311ae99dc65ffe96b9a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:07.788481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac859d29-a324-4b7f-9c9a-00ed9a8b5b7a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:07.899655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd81845-8f92-48fa-a746-ea88d97e8e7d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:07.903542 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-ac859d29-a324-4b7f-9c9a-00ed9a8b5b7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478b3b526a6a4311ae99dc65ffe96b9a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:07.919412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:07.924412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:07.924412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:07.924412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] PATH_INFO: `/v3/users/737b2c4d49344733a78c67dfcc1bb39d` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:07.930152 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=737b2c4d49344733a78c67dfcc1bb39d)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:07.934148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:07.944157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-ac859d29-a324-4b7f-9c9a-00ed9a8b5b7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478b3b526a6a4311ae99dc65ffe96b9a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:07.980511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-ac859d29-a324-4b7f-9c9a-00ed9a8b5b7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478b3b526a6a4311ae99dc65ffe96b9a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.004616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-ac859d29-a324-4b7f-9c9a-00ed9a8b5b7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478b3b526a6a4311ae99dc65ffe96b9a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.025913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-ac859d29-a324-4b7f-9c9a-00ed9a8b5b7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478b3b526a6a4311ae99dc65ffe96b9a'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.050426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1082/2152] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:07 2022] DELETE /identity/v3/users/478b3b526a6a4311ae99dc65ffe96b9a => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:08.081692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '737b2c4d49344733a78c67dfcc1bb39d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.112849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '737b2c4d49344733a78c67dfcc1bb39d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.113539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '737b2c4d49344733a78c67dfcc1bb39d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.114780 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '737b2c4d49344733a78c67dfcc1bb39d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.130903 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:08.139876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '737b2c4d49344733a78c67dfcc1bb39d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.197918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1071/2153] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:07 2022] DELETE /identity/v3/users/737b2c4d49344733a78c67dfcc1bb39d => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:08.203829 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaefa04-9e1e-4be4-8b12-3d85284b9f4a None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.204591 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaefa04-9e1e-4be4-8b12-3d85284b9f4a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.205043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaefa04-9e1e-4be4-8b12-3d85284b9f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.278004 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-feaefa04-9e1e-4be4-8b12-3d85284b9f4a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:08.284718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-feaefa04-9e1e-4be4-8b12-3d85284b9f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:08.386035 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1083/2154] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:08.399935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:08.401978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:08.531872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63ef60-3502-419c-81c2-45d8f34806e3 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:08.537932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf43d4f-3956-47c3-bb38-5223740bff71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:08.539105 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf43d4f-3956-47c3-bb38-5223740bff71 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.539613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf43d4f-3956-47c3-bb38-5223740bff71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.540202 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf43d4f-3956-47c3-bb38-5223740bff71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.553100 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf43d4f-3956-47c3-bb38-5223740bff71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:08.554714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf43d4f-3956-47c3-bb38-5223740bff71 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:08.563071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaefa04-9e1e-4be4-8b12-3d85284b9f4a None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:08.565675 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-feaefa04-9e1e-4be4-8b12-3d85284b9f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:08.595360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaefa04-9e1e-4be4-8b12-3d85284b9f4a None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:08.596103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1072/2155] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:08.601906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da65a3a-829b-48fb-b830-04f5413bf094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:08.603113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da65a3a-829b-48fb-b830-04f5413bf094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.603548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da65a3a-829b-48fb-b830-04f5413bf094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.603977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da65a3a-829b-48fb-b830-04f5413bf094 admin admin] PATH_INFO: `/v3/projects/7c4e3e64a3284cddbfd3f8134791dcf0` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.605593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da65a3a-829b-48fb-b830-04f5413bf094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c4e3e64a3284cddbfd3f8134791dcf0)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:08.608336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da65a3a-829b-48fb-b830-04f5413bf094 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:08.645173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:08.733121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2da65a3a-829b-48fb-b830-04f5413bf094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c4e3e64a3284cddbfd3f8134791dcf0', 'request_id': 'req-2da65a3a-829b-48fb-b830-04f5413bf094'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.747627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2da65a3a-829b-48fb-b830-04f5413bf094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c4e3e64a3284cddbfd3f8134791dcf0', 'request_id': 'req-2da65a3a-829b-48fb-b830-04f5413bf094'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.761271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2da65a3a-829b-48fb-b830-04f5413bf094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c4e3e64a3284cddbfd3f8134791dcf0', 'request_id': 'req-2da65a3a-829b-48fb-b830-04f5413bf094'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.786657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2da65a3a-829b-48fb-b830-04f5413bf094 admin admin] The token cache is being invalidate because project 7c4e3e64a3284cddbfd3f8134791dcf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:08.786960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2da65a3a-829b-48fb-b830-04f5413bf094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.787687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2da65a3a-829b-48fb-b830-04f5413bf094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:08.799976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1084/2156] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:08 2022] DELETE /identity/v3/projects/7c4e3e64a3284cddbfd3f8134791dcf0 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:08.801278 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7200f8d-38ef-4308-9091-7adcb15185ef None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.801426 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7200f8d-38ef-4308-9091-7adcb15185ef None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.801684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7200f8d-38ef-4308-9091-7adcb15185ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.834804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c7200f8d-38ef-4308-9091-7adcb15185ef None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:08.836791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c7200f8d-38ef-4308-9091-7adcb15185ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:08.995617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab17a05-9e11-41df-8313-ff6e1d15e7cb None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.995903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab17a05-9e11-41df-8313-ff6e1d15e7cb None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.996209 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab17a05-9e11-41df-8313-ff6e1d15e7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:09.006424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1073/2157] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:09.053472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-5ab17a05-9e11-41df-8313-ff6e1d15e7cb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:09.056276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5ab17a05-9e11-41df-8313-ff6e1d15e7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:09.081156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1085/2158] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:09.084857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7200f8d-38ef-4308-9091-7adcb15185ef None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:09.103066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7200f8d-38ef-4308-9091-7adcb15185ef None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:09.108915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29b531-ad70-4600-b3b6-5daf99995d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:09.111849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29b531-ad70-4600-b3b6-5daf99995d37 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:09.111849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29b531-ad70-4600-b3b6-5daf99995d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:09.112239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29b531-ad70-4600-b3b6-5daf99995d37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:09.112975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29b531-ad70-4600-b3b6-5daf99995d37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:09.113426 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29b531-ad70-4600-b3b6-5daf99995d37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:09.115180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29b531-ad70-4600-b3b6-5daf99995d37 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:09.138850 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab17a05-9e11-41df-8313-ff6e1d15e7cb None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:09.147119 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1074/2159] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:09.165987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab17a05-9e11-41df-8313-ff6e1d15e7cb None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:09.189403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebbba1-46a3-4428-9d9e-bedbfa1fd1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:09.193933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebbba1-46a3-4428-9d9e-bedbfa1fd1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:09.193933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebbba1-46a3-4428-9d9e-bedbfa1fd1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:09.194329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebbba1-46a3-4428-9d9e-bedbfa1fd1f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:09.197443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebbba1-46a3-4428-9d9e-bedbfa1fd1f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:09.202239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebbba1-46a3-4428-9d9e-bedbfa1fd1f7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:09.248605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29b531-ad70-4600-b3b6-5daf99995d37 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:09.254549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1a29b531-ad70-4600-b3b6-5daf99995d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:09.277833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1086/2160] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:09 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:09.290739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebbba1-46a3-4428-9d9e-bedbfa1fd1f7 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:09.336299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29b531-ad70-4600-b3b6-5daf99995d37 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:09.470876 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebbba1-46a3-4428-9d9e-bedbfa1fd1f7 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:09.480024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b1c2a-6860-4143-afff-54a222ccf07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:09.480823 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b1c2a-6860-4143-afff-54a222ccf07f admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:09.481623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b1c2a-6860-4143-afff-54a222ccf07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:09.481623 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b1c2a-6860-4143-afff-54a222ccf07f admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:09.482901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b1c2a-6860-4143-afff-54a222ccf07f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:09.496213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b1c2a-6860-4143-afff-54a222ccf07f admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:09.498603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fd932-d68b-490f-af62-bf861bf1c993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:09.501399 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fd932-d68b-490f-af62-bf861bf1c993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:09.502158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fd932-d68b-490f-af62-bf861bf1c993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:09.504059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fd932-d68b-490f-af62-bf861bf1c993 admin admin] PATH_INFO: `/v3/projects/aaac5ef7406349b298a290f8a1014c63` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:09.507717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fd932-d68b-490f-af62-bf861bf1c993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaac5ef7406349b298a290f8a1014c63)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:09.513717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fd932-d68b-490f-af62-bf861bf1c993 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:09.543714 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1087/2161] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:09 2022] POST /identity/v3/users => generated 291 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:09.548855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b1c2a-6860-4143-afff-54a222ccf07f admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:09.560270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:09.570143 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b1c2a-6860-4143-afff-54a222ccf07f admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:09.576796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eeaa2b-0f45-4ab6-8a9a-29e3ec84e590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:09.579809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eeaa2b-0f45-4ab6-8a9a-29e3ec84e590 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:09.579809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eeaa2b-0f45-4ab6-8a9a-29e3ec84e590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:09.580249 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eeaa2b-0f45-4ab6-8a9a-29e3ec84e590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:09.580690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eeaa2b-0f45-4ab6-8a9a-29e3ec84e590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:09.583238 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eeaa2b-0f45-4ab6-8a9a-29e3ec84e590 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:09.634953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1088/2162] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:09.636192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eeaa2b-0f45-4ab6-8a9a-29e3ec84e590 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:09.649120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-556fd932-d68b-490f-af62-bf861bf1c993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaac5ef7406349b298a290f8a1014c63', 'request_id': 'req-556fd932-d68b-490f-af62-bf861bf1c993'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:09.663729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eeaa2b-0f45-4ab6-8a9a-29e3ec84e590 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:09.665784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-556fd932-d68b-490f-af62-bf861bf1c993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaac5ef7406349b298a290f8a1014c63', 'request_id': 'req-556fd932-d68b-490f-af62-bf861bf1c993'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:09.697887 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-556fd932-d68b-490f-af62-bf861bf1c993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaac5ef7406349b298a290f8a1014c63', 'request_id': 'req-556fd932-d68b-490f-af62-bf861bf1c993'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:09.726119 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-556fd932-d68b-490f-af62-bf861bf1c993 admin admin] The token cache is being invalidate because project aaac5ef7406349b298a290f8a1014c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:09.727923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-556fd932-d68b-490f-af62-bf861bf1c993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:09.729348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-556fd932-d68b-490f-af62-bf861bf1c993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:09.732508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1075/2163] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:09 2022] DELETE /identity/v3/projects/aaac5ef7406349b298a290f8a1014c63 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:09.738594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8848e94-7908-4be6-8726-01bc4562451c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:09.739110 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8848e94-7908-4be6-8726-01bc4562451c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:09.739632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8848e94-7908-4be6-8726-01bc4562451c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:09.769713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:09.777318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890eaf10-fbce-48a2-95cd-20dfe4fc3aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:09.783337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-b8848e94-7908-4be6-8726-01bc4562451c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:09.785201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890eaf10-fbce-48a2-95cd-20dfe4fc3aad admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:09.786043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890eaf10-fbce-48a2-95cd-20dfe4fc3aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:09.788661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890eaf10-fbce-48a2-95cd-20dfe4fc3aad admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:09.789689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890eaf10-fbce-48a2-95cd-20dfe4fc3aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:09.793327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b8848e94-7908-4be6-8726-01bc4562451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:09.794921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890eaf10-fbce-48a2-95cd-20dfe4fc3aad admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:09.833426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1089/2164] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:09.838452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890eaf10-fbce-48a2-95cd-20dfe4fc3aad admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:09.842638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1076/2165] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:09.844301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-890eaf10-fbce-48a2-95cd-20dfe4fc3aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:09.849686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a70dcf-60d8-4ba4-a0b8-6b4a9bb61c64 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:09.849985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a70dcf-60d8-4ba4-a0b8-6b4a9bb61c64 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:09.850239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a70dcf-60d8-4ba4-a0b8-6b4a9bb61c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:09.868250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890eaf10-fbce-48a2-95cd-20dfe4fc3aad admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:09.879069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f34b8-4cd5-4a72-9aa9-c5b237cd00e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:09.881067 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f34b8-4cd5-4a72-9aa9-c5b237cd00e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:09.882080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f34b8-4cd5-4a72-9aa9-c5b237cd00e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:09.882582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f34b8-4cd5-4a72-9aa9-c5b237cd00e4 admin admin] PATH_INFO: `/v3/projects/8b875238d84845bc852e14451f4061ce/users/4ad6163e37144bccbdb0a51cc0f1f09c/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:09.907668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-74a70dcf-60d8-4ba4-a0b8-6b4a9bb61c64 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:09.909140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-74a70dcf-60d8-4ba4-a0b8-6b4a9bb61c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:09.918810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f34b8-4cd5-4a72-9aa9-c5b237cd00e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b875238d84845bc852e14451f4061ce, user_id=4ad6163e37144bccbdb0a51cc0f1f09c, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:09.921903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f34b8-4cd5-4a72-9aa9-c5b237cd00e4 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:09.922946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1077/2166] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:09.924163 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a70dcf-60d8-4ba4-a0b8-6b4a9bb61c64 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:09.948478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1090/2167] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:09 2022] PUT /identity/v3/projects/8b875238d84845bc852e14451f4061ce/users/4ad6163e37144bccbdb0a51cc0f1f09c/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:09.960320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fdc67-595e-4c98-9f82-b0b51de14219 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:09.960589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fdc67-595e-4c98-9f82-b0b51de14219 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:09.960861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fdc67-595e-4c98-9f82-b0b51de14219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:09.965899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a70dcf-60d8-4ba4-a0b8-6b4a9bb61c64 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:10.025936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b06fdc67-595e-4c98-9f82-b0b51de14219 None None] MFA Rules not processed for user `4ad6163e37144bccbdb0a51cc0f1f09c`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:10.027259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b06fdc67-595e-4c98-9f82-b0b51de14219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:10.135185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998910b-9743-43f4-8047-de59b5c9a034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:10.135848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998910b-9743-43f4-8047-de59b5c9a034 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:10.140160 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998910b-9743-43f4-8047-de59b5c9a034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:10.140537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998910b-9743-43f4-8047-de59b5c9a034 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:10.142332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998910b-9743-43f4-8047-de59b5c9a034 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:10.142783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998910b-9743-43f4-8047-de59b5c9a034 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:10.151374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998910b-9743-43f4-8047-de59b5c9a034 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:10.199908 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1078/2168] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:10.201232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998910b-9743-43f4-8047-de59b5c9a034 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:10.225293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998910b-9743-43f4-8047-de59b5c9a034 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:10.242360 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c3d72-eb17-4b8a-af31-20be6dc8378b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:10.242999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c3d72-eb17-4b8a-af31-20be6dc8378b admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:10.243438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c3d72-eb17-4b8a-af31-20be6dc8378b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:10.244005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c3d72-eb17-4b8a-af31-20be6dc8378b admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:10.244707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c3d72-eb17-4b8a-af31-20be6dc8378b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:10.251053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c3d72-eb17-4b8a-af31-20be6dc8378b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:10.267117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1091/2169] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:10.268983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fdc67-595e-4c98-9f82-b0b51de14219 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:10.289638 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fdc67-595e-4c98-9f82-b0b51de14219 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:10.295751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ea090-578a-46bc-9f13-376d30d87ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:10.296949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ea090-578a-46bc-9f13-376d30d87ccf admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:10.297613 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ea090-578a-46bc-9f13-376d30d87ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:10.298105 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ea090-578a-46bc-9f13-376d30d87ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:10.299110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ea090-578a-46bc-9f13-376d30d87ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:10.302544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ea090-578a-46bc-9f13-376d30d87ccf admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:10.324117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1079/2170] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:10 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:10.329831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c3d72-eb17-4b8a-af31-20be6dc8378b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:10.366181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1092/2171] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:10 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:10.375015 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ea090-578a-46bc-9f13-376d30d87ccf admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:10.478700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c3d72-eb17-4b8a-af31-20be6dc8378b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:10.484652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:10.543263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ea090-578a-46bc-9f13-376d30d87ccf admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:10.556815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6396f-54c1-4187-9b6b-343465b13259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:10.557911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6396f-54c1-4187-9b6b-343465b13259 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:10.559156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6396f-54c1-4187-9b6b-343465b13259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:10.559762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6396f-54c1-4187-9b6b-343465b13259 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:10.561699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6396f-54c1-4187-9b6b-343465b13259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:10.566407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6396f-54c1-4187-9b6b-343465b13259 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:10.581913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ede3a8-260b-4a8b-ab75-dd92f2d4d868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:10.584747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ede3a8-260b-4a8b-ab75-dd92f2d4d868 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:10.585284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ede3a8-260b-4a8b-ab75-dd92f2d4d868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:10.585684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ede3a8-260b-4a8b-ab75-dd92f2d4d868 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:10.586588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ede3a8-260b-4a8b-ab75-dd92f2d4d868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:10.589624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ede3a8-260b-4a8b-ab75-dd92f2d4d868 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:10.638256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1093/2172] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:10 2022] POST /identity/v3/users => generated 297 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:10.646650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6396f-54c1-4187-9b6b-343465b13259 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:10.659473 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1080/2173] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:10 2022] POST /identity/v3/users => generated 292 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:10.660616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ede3a8-260b-4a8b-ab75-dd92f2d4d868 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:10.661209 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6396f-54c1-4187-9b6b-343465b13259 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:10.667137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742bfaa-46b0-4d04-8be4-4fb4a686f4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:10.667965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742bfaa-46b0-4d04-8be4-4fb4a686f4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:10.668288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742bfaa-46b0-4d04-8be4-4fb4a686f4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:10.668608 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742bfaa-46b0-4d04-8be4-4fb4a686f4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:10.670680 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3742bfaa-46b0-4d04-8be4-4fb4a686f4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:10.674120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ede3a8-260b-4a8b-ab75-dd92f2d4d868 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:10.674932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3742bfaa-46b0-4d04-8be4-4fb4a686f4b7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:10.679937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9b22f-ec04-448a-9b69-ec5ef255468d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:10.680931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9b22f-ec04-448a-9b69-ec5ef255468d admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:10.681307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9b22f-ec04-448a-9b69-ec5ef255468d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:10.681677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9b22f-ec04-448a-9b69-ec5ef255468d admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:10.682442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9b22f-ec04-448a-9b69-ec5ef255468d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:10.684651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9b22f-ec04-448a-9b69-ec5ef255468d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:10.691169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1094/2174] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:10.696684 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742bfaa-46b0-4d04-8be4-4fb4a686f4b7 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:10.701138 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1081/2175] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:10.705999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9b22f-ec04-448a-9b69-ec5ef255468d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:10.717704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742bfaa-46b0-4d04-8be4-4fb4a686f4b7 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:10.719366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9b22f-ec04-448a-9b69-ec5ef255468d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:10.726117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f7731-46e7-4f9f-954b-4430df163e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:10.727237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f7731-46e7-4f9f-954b-4430df163e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:10.727561 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae045f-f97d-4310-aa68-25a72b2b3983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:10.727946 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f7731-46e7-4f9f-954b-4430df163e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:10.728385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae045f-f97d-4310-aa68-25a72b2b3983 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:10.728736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae045f-f97d-4310-aa68-25a72b2b3983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:10.728843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f7731-46e7-4f9f-954b-4430df163e20 admin admin] PATH_INFO: `/v3/projects/1cefd9bb46dd4721b27501b46917c42d/users/6db0027cb9c340f590b0882c4b6cbd30/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:10.729132 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae045f-f97d-4310-aa68-25a72b2b3983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:10.729868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae045f-f97d-4310-aa68-25a72b2b3983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:10.732014 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae045f-f97d-4310-aa68-25a72b2b3983 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:10.746978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:10.749252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1082/2176] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:10.754350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f7731-46e7-4f9f-954b-4430df163e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cefd9bb46dd4721b27501b46917c42d, user_id=6db0027cb9c340f590b0882c4b6cbd30, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:10.755723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae045f-f97d-4310-aa68-25a72b2b3983 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:10.761172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f7731-46e7-4f9f-954b-4430df163e20 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:10.769847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae045f-f97d-4310-aa68-25a72b2b3983 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:10.784661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508e5b4-107c-4f56-8378-08c60f1ccd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:10.785651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508e5b4-107c-4f56-8378-08c60f1ccd02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:10.787305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508e5b4-107c-4f56-8378-08c60f1ccd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:10.787745 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508e5b4-107c-4f56-8378-08c60f1ccd02 admin admin] PATH_INFO: `/v3/projects/2815adc7e64040c8986b1bd3f4519343/users/64b579837fb4434282224eb7b83e57ab/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:10.800531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1095/2177] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:10 2022] PUT /identity/v3/projects/1cefd9bb46dd4721b27501b46917c42d/users/6db0027cb9c340f590b0882c4b6cbd30/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:10.801879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f7731-46e7-4f9f-954b-4430df163e20 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:10.824636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508e5b4-107c-4f56-8378-08c60f1ccd02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2815adc7e64040c8986b1bd3f4519343, user_id=64b579837fb4434282224eb7b83e57ab, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:10.827581 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508e5b4-107c-4f56-8378-08c60f1ccd02 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:10.883156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1083/2178] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:10 2022] PUT /identity/v3/projects/2815adc7e64040c8986b1bd3f4519343/users/64b579837fb4434282224eb7b83e57ab/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:10.897368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18c9c-9c09-46fe-a1ea-4eeb98ad6244 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:10.897803 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18c9c-9c09-46fe-a1ea-4eeb98ad6244 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:10.897985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18c9c-9c09-46fe-a1ea-4eeb98ad6244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:10.956326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f7731-46e7-4f9f-954b-4430df163e20 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:10.959845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c7d18c9c-9c09-46fe-a1ea-4eeb98ad6244 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:10.961311 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c7d18c9c-9c09-46fe-a1ea-4eeb98ad6244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:11.076793 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1084/2179] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:11.087798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d18c9c-9c09-46fe-a1ea-4eeb98ad6244 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:11.089902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c7d18c9c-9c09-46fe-a1ea-4eeb98ad6244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:11.091531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65394430-e7de-48e7-800f-f83c55e2744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:11.092465 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65394430-e7de-48e7-800f-f83c55e2744a admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:11.093300 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65394430-e7de-48e7-800f-f83c55e2744a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:11.095533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65394430-e7de-48e7-800f-f83c55e2744a admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:11.101448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65394430-e7de-48e7-800f-f83c55e2744a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:11.103215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65394430-e7de-48e7-800f-f83c55e2744a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:11.122034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1096/2180] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:11.129392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65394430-e7de-48e7-800f-f83c55e2744a admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:11.146578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65394430-e7de-48e7-800f-f83c55e2744a admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:11.156207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c360e-ee53-42b8-a971-39ba2a43e6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:11.158484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c360e-ee53-42b8-a971-39ba2a43e6b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:11.159136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c360e-ee53-42b8-a971-39ba2a43e6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:11.159426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c360e-ee53-42b8-a971-39ba2a43e6b7 admin admin] PATH_INFO: `/v3/projects/1cefd9bb46dd4721b27501b46917c42d/users/6db0027cb9c340f590b0882c4b6cbd30/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:11.162541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c360e-ee53-42b8-a971-39ba2a43e6b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cefd9bb46dd4721b27501b46917c42d, user_id=6db0027cb9c340f590b0882c4b6cbd30, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:11.167011 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c360e-ee53-42b8-a971-39ba2a43e6b7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:11.183714 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1097/2181] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:11 2022] PUT /identity/v3/projects/1cefd9bb46dd4721b27501b46917c42d/users/6db0027cb9c340f590b0882c4b6cbd30/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:11.191266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d18c9c-9c09-46fe-a1ea-4eeb98ad6244 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:11.195406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beaecc-17c5-43cb-9a20-439bc7e61c30 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:11.195690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beaecc-17c5-43cb-9a20-439bc7e61c30 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:11.195965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beaecc-17c5-43cb-9a20-439bc7e61c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:11.198879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:11.233507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b1beaecc-17c5-43cb-9a20-439bc7e61c30 None None] MFA Rules not processed for user `6db0027cb9c340f590b0882c4b6cbd30`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:11.236862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b1beaecc-17c5-43cb-9a20-439bc7e61c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:11.291290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df29049-d3d1-4082-9b03-1ee4936fbff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:11.292385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df29049-d3d1-4082-9b03-1ee4936fbff9 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:11.292981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df29049-d3d1-4082-9b03-1ee4936fbff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:11.294308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df29049-d3d1-4082-9b03-1ee4936fbff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:11.379045 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1098/2182] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:11.394275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beaecc-17c5-43cb-9a20-439bc7e61c30 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:11.396589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b1beaecc-17c5-43cb-9a20-439bc7e61c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:11.423380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:11.446650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df29049-d3d1-4082-9b03-1ee4936fbff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:11.449762 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df29049-d3d1-4082-9b03-1ee4936fbff9 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:11.470882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1085/2183] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:11.538012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beaecc-17c5-43cb-9a20-439bc7e61c30 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:11.545112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e666512-a1e3-4f9d-bb60-ba827b66bf52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:11.546752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e666512-a1e3-4f9d-bb60-ba827b66bf52 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:11.546961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e666512-a1e3-4f9d-bb60-ba827b66bf52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:11.547446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e666512-a1e3-4f9d-bb60-ba827b66bf52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:11.570824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e666512-a1e3-4f9d-bb60-ba827b66bf52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:11.575101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e666512-a1e3-4f9d-bb60-ba827b66bf52 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:11.600108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1099/2184] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:11.748860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df29049-d3d1-4082-9b03-1ee4936fbff9 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:11.754040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e666512-a1e3-4f9d-bb60-ba827b66bf52 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:11.755061 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0df29049-d3d1-4082-9b03-1ee4936fbff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:11.764958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e666512-a1e3-4f9d-bb60-ba827b66bf52 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:11.770365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b40db-4fb1-4c1b-8be9-cb66f9ef28d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:11.771587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b40db-4fb1-4c1b-8be9-cb66f9ef28d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:11.772261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b40db-4fb1-4c1b-8be9-cb66f9ef28d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:11.772888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b40db-4fb1-4c1b-8be9-cb66f9ef28d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:11.784804 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df29049-d3d1-4082-9b03-1ee4936fbff9 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:11.815514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:11.816979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:11.817684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:11.818354 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] PATH_INFO: `/v3/users/aa0b29fa73cf48a1bec3253f89f01377` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:11.821173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0b29fa73cf48a1bec3253f89f01377)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:11.825000 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:11.854771 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b40db-4fb1-4c1b-8be9-cb66f9ef28d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:11.856617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b40db-4fb1-4c1b-8be9-cb66f9ef28d8 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:11.868505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1100/2185] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:11.961580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0b29fa73cf48a1bec3253f89f01377'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:11.989641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0b29fa73cf48a1bec3253f89f01377'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:11.990460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0b29fa73cf48a1bec3253f89f01377'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:11.991204 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0b29fa73cf48a1bec3253f89f01377'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:12.029915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0b29fa73cf48a1bec3253f89f01377'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:12.050372 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1086/2186] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:11 2022] DELETE /identity/v3/users/aa0b29fa73cf48a1bec3253f89f01377 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:12.057167 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c952e2-2322-44b5-a565-c0c07d7e033e None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:12.057815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c952e2-2322-44b5-a565-c0c07d7e033e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:12.058194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c952e2-2322-44b5-a565-c0c07d7e033e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:12.096221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-a2c952e2-2322-44b5-a565-c0c07d7e033e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:12.098180 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a2c952e2-2322-44b5-a565-c0c07d7e033e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:12.217988 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1101/2187] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:12.232570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:12.234570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:12.325396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:12.333392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c952e2-2322-44b5-a565-c0c07d7e033e None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:12.335087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a2c952e2-2322-44b5-a565-c0c07d7e033e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:12.351479 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c952e2-2322-44b5-a565-c0c07d7e033e None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:12.357373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:12.358675 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:12.358964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:12.359240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] PATH_INFO: `/v3/users/4ad6163e37144bccbdb0a51cc0f1f09c` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:12.363055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad6163e37144bccbdb0a51cc0f1f09c)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:12.363432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9adbbe-b4c1-402a-9575-6f5798a82e65 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:12.366907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:12.367400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa952f-6356-47a0-a4c0-8dcccbacf0bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:12.368111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa952f-6356-47a0-a4c0-8dcccbacf0bf service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:12.368356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa952f-6356-47a0-a4c0-8dcccbacf0bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:12.368582 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa952f-6356-47a0-a4c0-8dcccbacf0bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:12.379653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa952f-6356-47a0-a4c0-8dcccbacf0bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:12.381156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa952f-6356-47a0-a4c0-8dcccbacf0bf service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:12.392466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1087/2188] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:12.434278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:12.440531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad6163e37144bccbdb0a51cc0f1f09c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:12.459413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad6163e37144bccbdb0a51cc0f1f09c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:12.472116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad6163e37144bccbdb0a51cc0f1f09c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:12.481572 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad6163e37144bccbdb0a51cc0f1f09c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:12.482331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad6163e37144bccbdb0a51cc0f1f09c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:12.492094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1102/2189] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:12 2022] DELETE /identity/v3/users/4ad6163e37144bccbdb0a51cc0f1f09c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:12.497710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b92141-4f35-489e-a19f-513cdc6d6cae None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:12.498202 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b92141-4f35-489e-a19f-513cdc6d6cae None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:12.498814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b92141-4f35-489e-a19f-513cdc6d6cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:12.543904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-90b92141-4f35-489e-a19f-513cdc6d6cae None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:12.545670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-90b92141-4f35-489e-a19f-513cdc6d6cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:12.667125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1088/2190] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:12.727706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:12.731894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:12.871470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cab1f1-9455-401d-9e14-749aa2bbd25b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:12.887462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05be12-36cd-4d3f-8bcd-bb535cf19d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:12.888596 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05be12-36cd-4d3f-8bcd-bb535cf19d75 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:12.889084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05be12-36cd-4d3f-8bcd-bb535cf19d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:12.889568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05be12-36cd-4d3f-8bcd-bb535cf19d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:12.910264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05be12-36cd-4d3f-8bcd-bb535cf19d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:12.912435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05be12-36cd-4d3f-8bcd-bb535cf19d75 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:12.941629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1103/2191] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:13.331813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b92141-4f35-489e-a19f-513cdc6d6cae None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:13.335128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-90b92141-4f35-489e-a19f-513cdc6d6cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:13.365335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b92141-4f35-489e-a19f-513cdc6d6cae None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:13.373621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:13.374378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:13.374755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:13.375128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] PATH_INFO: `/v3/projects/1cf4f0c64dad45a9a46f00590f2c24c5` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:13.378581 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf4f0c64dad45a9a46f00590f2c24c5)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:13.381863 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:13.587624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf4f0c64dad45a9a46f00590f2c24c5', 'request_id': 'req-61b19577-780c-4aca-a88c-ce15a6f655ca'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:13.605085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf4f0c64dad45a9a46f00590f2c24c5', 'request_id': 'req-61b19577-780c-4aca-a88c-ce15a6f655ca'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:13.624473 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf4f0c64dad45a9a46f00590f2c24c5', 'request_id': 'req-61b19577-780c-4aca-a88c-ce15a6f655ca'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:13.663815 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] The token cache is being invalidate because project 1cf4f0c64dad45a9a46f00590f2c24c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:13.664167 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:13.666754 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:13.667858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a82972-a375-40f4-a60d-7513ca9dc667 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:13.668179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a82972-a375-40f4-a60d-7513ca9dc667 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:13.668488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a82972-a375-40f4-a60d-7513ca9dc667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:13.669991 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1089/2192] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:13 2022] DELETE /identity/v3/projects/1cf4f0c64dad45a9a46f00590f2c24c5 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:13.675310 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:13.679383 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:13.723770 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-78a82972-a375-40f4-a60d-7513ca9dc667 None None] MFA Rules not processed for user `64b579837fb4434282224eb7b83e57ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:13.723770 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-78a82972-a375-40f4-a60d-7513ca9dc667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:13.850635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:13.852087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1104/2193] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:13.861541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b19577-780c-4aca-a88c-ce15a6f655ca admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:13.866690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a82972-a375-40f4-a60d-7513ca9dc667 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:13.869085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:13.870394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:13.871113 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:13.872391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] PATH_INFO: `/v3/users/060c483874694eb6867d461f3e0dbe81` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:13.872816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-78a82972-a375-40f4-a60d-7513ca9dc667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:13.878055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=060c483874694eb6867d461f3e0dbe81)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:13.882538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:14.043518 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '060c483874694eb6867d461f3e0dbe81'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:14.053626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a82972-a375-40f4-a60d-7513ca9dc667 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:14.061194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cc098-b956-499b-980b-d35d16e6fa41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:14.062451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cc098-b956-499b-980b-d35d16e6fa41 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:14.063166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cc098-b956-499b-980b-d35d16e6fa41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:14.063774 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cc098-b956-499b-980b-d35d16e6fa41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:14.069570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '060c483874694eb6867d461f3e0dbe81'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:14.074189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '060c483874694eb6867d461f3e0dbe81'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:14.077657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '060c483874694eb6867d461f3e0dbe81'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:14.084156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:14.090160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2cc098-b956-499b-980b-d35d16e6fa41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:14.095933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2cc098-b956-499b-980b-d35d16e6fa41 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:14.118114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '060c483874694eb6867d461f3e0dbe81'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:14.133622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1105/2194] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:14.189370 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1090/2195] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:13 2022] DELETE /identity/v3/users/060c483874694eb6867d461f3e0dbe81 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:14.693615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cc098-b956-499b-980b-d35d16e6fa41 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:14.697373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6c2cc098-b956-499b-980b-d35d16e6fa41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:14.781719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cc098-b956-499b-980b-d35d16e6fa41 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:15.021879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:15.021879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:15.021879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:15.021879 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] PATH_INFO: `/v3/projects/8b875238d84845bc852e14451f4061ce` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:15.025482 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b875238d84845bc852e14451f4061ce)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:15.027735 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:15.264424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b875238d84845bc852e14451f4061ce', 'request_id': 'req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:15.285057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b875238d84845bc852e14451f4061ce', 'request_id': 'req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:15.295250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b875238d84845bc852e14451f4061ce', 'request_id': 'req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:15.347979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] The token cache is being invalidate because project 8b875238d84845bc852e14451f4061ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:15.352325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:15.353999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:15.366012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1106/2196] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:14 2022] DELETE /identity/v3/projects/8b875238d84845bc852e14451f4061ce => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:15.366382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:15.369573 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:15.443892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344698f-8d3c-4ddc-87e4-ee5bf3e38245 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:15.593770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2340bc-07b7-4b9e-bb71-aeaa1667180c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:15.598117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2340bc-07b7-4b9e-bb71-aeaa1667180c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:15.603038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2340bc-07b7-4b9e-bb71-aeaa1667180c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:15.635316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2340bc-07b7-4b9e-bb71-aeaa1667180c admin admin] PATH_INFO: `/v3/users/6db0027cb9c340f590b0882c4b6cbd30` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:15.644779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2340bc-07b7-4b9e-bb71-aeaa1667180c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6db0027cb9c340f590b0882c4b6cbd30)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:15.649478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2340bc-07b7-4b9e-bb71-aeaa1667180c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:15.808678 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-7c2340bc-07b7-4b9e-bb71-aeaa1667180c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6db0027cb9c340f590b0882c4b6cbd30'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:15.832958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-7c2340bc-07b7-4b9e-bb71-aeaa1667180c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6db0027cb9c340f590b0882c4b6cbd30'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:15.835045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-7c2340bc-07b7-4b9e-bb71-aeaa1667180c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6db0027cb9c340f590b0882c4b6cbd30'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:15.835889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-7c2340bc-07b7-4b9e-bb71-aeaa1667180c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6db0027cb9c340f590b0882c4b6cbd30'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:15.882461 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-7c2340bc-07b7-4b9e-bb71-aeaa1667180c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6db0027cb9c340f590b0882c4b6cbd30'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:15.891838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:15.925197 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1091/2197] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:15 2022] DELETE /identity/v3/users/6db0027cb9c340f590b0882c4b6cbd30 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:16.093548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:16.098188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:16.138888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d55fb6-9e7f-4a58-b54f-cef782fa72c7 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:16.166930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b26902-4c72-4642-bb78-6ae4fae37c00 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:16.167307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b26902-4c72-4642-bb78-6ae4fae37c00 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:16.167591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b26902-4c72-4642-bb78-6ae4fae37c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:16.191328 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-82b26902-4c72-4642-bb78-6ae4fae37c00 None None] MFA Rules not processed for user `64b579837fb4434282224eb7b83e57ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:16.192626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-82b26902-4c72-4642-bb78-6ae4fae37c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:16.325851 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:16.326571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:16.327044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:16.327538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1092/2198] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:16.327839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] PATH_INFO: `/v3/projects/220f3c73702f457aa101e3df678c4c8f` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:16.330101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=220f3c73702f457aa101e3df678c4c8f)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:16.332985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:16.383038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:16.414008 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '220f3c73702f457aa101e3df678c4c8f', 'request_id': 'req-dcbebe03-e8c6-4fd1-8947-7c093b684e05'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.432999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '220f3c73702f457aa101e3df678c4c8f', 'request_id': 'req-dcbebe03-e8c6-4fd1-8947-7c093b684e05'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.455755 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '220f3c73702f457aa101e3df678c4c8f', 'request_id': 'req-dcbebe03-e8c6-4fd1-8947-7c093b684e05'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.475897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] The token cache is being invalidate because project 220f3c73702f457aa101e3df678c4c8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:16.476307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.477058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.480050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1107/2199] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:16 2022] DELETE /identity/v3/projects/220f3c73702f457aa101e3df678c4c8f => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:16.485073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b508-759e-4d6e-9c6e-d5a7ede34a85 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:16.486282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b508-759e-4d6e-9c6e-d5a7ede34a85 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:16.486805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b508-759e-4d6e-9c6e-d5a7ede34a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:16.529098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-6108b508-759e-4d6e-9c6e-d5a7ede34a85 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:16.530701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6108b508-759e-4d6e-9c6e-d5a7ede34a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:16.537013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:16.538609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:16.713533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbebe03-e8c6-4fd1-8947-7c093b684e05 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:16.730599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:16.736180 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:16.736720 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:16.737000 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] PATH_INFO: `/v3/users/8183e1619775451283cb6fec3935243b` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:16.744400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8183e1619775451283cb6fec3935243b)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:16.746483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:16.753430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1093/2200] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:16.760222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a800c-149f-418a-823b-e26db65a26ce None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:16.760603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a800c-149f-418a-823b-e26db65a26ce None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:16.760849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a800c-149f-418a-823b-e26db65a26ce None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:16.762641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1094/2201] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:18:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:18:16.764126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a800c-149f-418a-823b-e26db65a26ce None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:16.766724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b06a800c-149f-418a-823b-e26db65a26ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:16.790124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a800c-149f-418a-823b-e26db65a26ce None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:16.796097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f1f99-7233-44b7-b917-4735fd92e20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:16.796761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f1f99-7233-44b7-b917-4735fd92e20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:16.797636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f1f99-7233-44b7-b917-4735fd92e20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:16.797976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f1f99-7233-44b7-b917-4735fd92e20f admin admin] PATH_INFO: `/v3/projects/1cefd9bb46dd4721b27501b46917c42d` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:16.799637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f1f99-7233-44b7-b917-4735fd92e20f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cefd9bb46dd4721b27501b46917c42d)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:16.802175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f1f99-7233-44b7-b917-4735fd92e20f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:16.874248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3b5f1f99-7233-44b7-b917-4735fd92e20f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cefd9bb46dd4721b27501b46917c42d', 'request_id': 'req-3b5f1f99-7233-44b7-b917-4735fd92e20f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.874813 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8183e1619775451283cb6fec3935243b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.892422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3b5f1f99-7233-44b7-b917-4735fd92e20f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cefd9bb46dd4721b27501b46917c42d', 'request_id': 'req-3b5f1f99-7233-44b7-b917-4735fd92e20f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.900779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3b5f1f99-7233-44b7-b917-4735fd92e20f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cefd9bb46dd4721b27501b46917c42d', 'request_id': 'req-3b5f1f99-7233-44b7-b917-4735fd92e20f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.907225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8183e1619775451283cb6fec3935243b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.923866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3b5f1f99-7233-44b7-b917-4735fd92e20f admin admin] The token cache is being invalidate because project 1cefd9bb46dd4721b27501b46917c42d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:16.924156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8183e1619775451283cb6fec3935243b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.924634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3b5f1f99-7233-44b7-b917-4735fd92e20f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.926505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3b5f1f99-7233-44b7-b917-4735fd92e20f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.928685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1095/2202] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:16 2022] DELETE /identity/v3/projects/1cefd9bb46dd4721b27501b46917c42d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:16.931142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a818e6-b352-41c3-be71-ba8f65930f0f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:16.931637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a818e6-b352-41c3-be71-ba8f65930f0f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:16.932205 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a818e6-b352-41c3-be71-ba8f65930f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:16.945833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8183e1619775451283cb6fec3935243b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.947007 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8183e1619775451283cb6fec3935243b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.959941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-b7a818e6-b352-41c3-be71-ba8f65930f0f None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:16.960331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1108/2203] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:16 2022] DELETE /identity/v3/users/8183e1619775451283cb6fec3935243b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:16.961462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b7a818e6-b352-41c3-be71-ba8f65930f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:16.961961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:16.964640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.013774 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:17.073394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:17.082288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1096/2204] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:17.084295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8befb9-b747-4935-87fc-c84d34cba0ae None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:17.084890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8befb9-b747-4935-87fc-c84d34cba0ae None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:17.085519 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8befb9-b747-4935-87fc-c84d34cba0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:17.115281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-df8befb9-b747-4935-87fc-c84d34cba0ae None None] MFA Rules not processed for user `08f34295f19b4d6ea405f1e8c8190db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:17.116642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-df8befb9-b747-4935-87fc-c84d34cba0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.130691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83cbf9-2eee-4da0-8026-639dfadbafcc admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:17.152833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:17.153367 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:17.153682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:17.153998 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] PATH_INFO: `/v3/users/5b2e7c46b6aa4800834115196866b244` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:17.155292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2e7c46b6aa4800834115196866b244)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:17.158543 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:17.202498 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1097/2205] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:18:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:17.204375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e3355-ec27-4d43-966f-be1de09ac0a6 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:17.204890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e3355-ec27-4d43-966f-be1de09ac0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:17.205030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e3355-ec27-4d43-966f-be1de09ac0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:17.233973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2e7c46b6aa4800834115196866b244'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:17.236897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-aa2e3355-ec27-4d43-966f-be1de09ac0a6 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:17.238783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-aa2e3355-ec27-4d43-966f-be1de09ac0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.252401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2e7c46b6aa4800834115196866b244'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:17.253758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1098/2206] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:17.255544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b904d5-065b-4fb3-b005-0f2e85aecb58 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:17.255836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b904d5-065b-4fb3-b005-0f2e85aecb58 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:17.256147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b904d5-065b-4fb3-b005-0f2e85aecb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:17.264767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2e7c46b6aa4800834115196866b244'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:17.277647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2e7c46b6aa4800834115196866b244'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:17.278689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2e7c46b6aa4800834115196866b244'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:17.288891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1109/2207] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:16 2022] DELETE /identity/v3/users/5b2e7c46b6aa4800834115196866b244 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:17.289111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-a3b904d5-065b-4fb3-b005-0f2e85aecb58 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:17.289902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:17.290895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a3b904d5-065b-4fb3-b005-0f2e85aecb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.291887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.467228 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc94bec-92d3-4c7a-bbcb-1e2a259297e8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:17.476607 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50ddf0-693c-426c-a9dd-38b7a092ec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:17.478451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50ddf0-693c-426c-a9dd-38b7a092ec2a admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:17.478741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50ddf0-693c-426c-a9dd-38b7a092ec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:17.479479 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50ddf0-693c-426c-a9dd-38b7a092ec2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:17.480748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50ddf0-693c-426c-a9dd-38b7a092ec2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:17.481054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50ddf0-693c-426c-a9dd-38b7a092ec2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:17.482960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50ddf0-693c-426c-a9dd-38b7a092ec2a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:17.486919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1099/2208] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:17.487778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b904d5-065b-4fb3-b005-0f2e85aecb58 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:17.490096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a3b904d5-065b-4fb3-b005-0f2e85aecb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.518932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1110/2209] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:17.520456 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423573-5cd7-4b6f-a79c-be197f37997b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:17.520741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423573-5cd7-4b6f-a79c-be197f37997b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:17.521014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423573-5cd7-4b6f-a79c-be197f37997b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:17.550192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-1d423573-5cd7-4b6f-a79c-be197f37997b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:17.551589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1d423573-5cd7-4b6f-a79c-be197f37997b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.561443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1111/2210] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:17.563531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5e2aa-17dd-488c-99be-c8705dfa09a9 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:17.564238 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5e2aa-17dd-488c-99be-c8705dfa09a9 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:17.564880 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5e2aa-17dd-488c-99be-c8705dfa09a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:17.592542 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-80d5e2aa-17dd-488c-99be-c8705dfa09a9 None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:17.594103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-80d5e2aa-17dd-488c-99be-c8705dfa09a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.653002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:17.678905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:17.704050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b904d5-065b-4fb3-b005-0f2e85aecb58 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:17.714938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73aafd-0cb0-4660-9202-6dfd9cec96a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08f34295f19b4d6ea405f1e8c8190db5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:17.716035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73aafd-0cb0-4660-9202-6dfd9cec96a8 service placement] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:17.716619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73aafd-0cb0-4660-9202-6dfd9cec96a8 service placement] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:17.717129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73aafd-0cb0-4660-9202-6dfd9cec96a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:17.720814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7d73aafd-0cb0-4660-9202-6dfd9cec96a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.743321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1112/2211] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:18:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:17.744866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5e2aa-17dd-488c-99be-c8705dfa09a9 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:17.750839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-80d5e2aa-17dd-488c-99be-c8705dfa09a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.785968 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73aafd-0cb0-4660-9202-6dfd9cec96a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:17.789509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73aafd-0cb0-4660-9202-6dfd9cec96a8 service placement] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:17.815805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1100/2212] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:17.817132 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73aafd-0cb0-4660-9202-6dfd9cec96a8 service placement] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:17.840303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73aafd-0cb0-4660-9202-6dfd9cec96a8 service placement] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:17.849634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11fb4c-cf9e-494b-bbc2-d03ed5ae9833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:17.850632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11fb4c-cf9e-494b-bbc2-d03ed5ae9833 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:17.851116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11fb4c-cf9e-494b-bbc2-d03ed5ae9833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:17.851665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11fb4c-cf9e-494b-bbc2-d03ed5ae9833 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:17.852990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e11fb4c-cf9e-494b-bbc2-d03ed5ae9833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:17.856944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e11fb4c-cf9e-494b-bbc2-d03ed5ae9833 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:17.893573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5e2aa-17dd-488c-99be-c8705dfa09a9 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:17.896983 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b32728-46d6-4e98-9c4a-6ee138685bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:17.897568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b32728-46d6-4e98-9c4a-6ee138685bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:17.897840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b32728-46d6-4e98-9c4a-6ee138685bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:17.898099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b32728-46d6-4e98-9c4a-6ee138685bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:17.910085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b32728-46d6-4e98-9c4a-6ee138685bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:17.911728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b32728-46d6-4e98-9c4a-6ee138685bc6 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:17.920501 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1101/2213] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:17 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:17.921506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11fb4c-cf9e-494b-bbc2-d03ed5ae9833 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:17.927741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1113/2214] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:17.928669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b32728-46d6-4e98-9c4a-6ee138685bc6 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:17.930316 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e2b32728-46d6-4e98-9c4a-6ee138685bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.950302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b32728-46d6-4e98-9c4a-6ee138685bc6 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:18.058446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:18.059823 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:18.060360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:18.060900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] PATH_INFO: `/v3/projects/9bf90b4e09e94cc984fe36b03b08a0b6` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:18.064025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf90b4e09e94cc984fe36b03b08a0b6)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:18.068143 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:18.084664 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11fb4c-cf9e-494b-bbc2-d03ed5ae9833 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:18.096972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000a9f0-3b5f-40f6-890c-40a7c62e7615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:18.098276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000a9f0-3b5f-40f6-890c-40a7c62e7615 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:18.099911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000a9f0-3b5f-40f6-890c-40a7c62e7615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:18.100459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000a9f0-3b5f-40f6-890c-40a7c62e7615 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:18.101697 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000a9f0-3b5f-40f6-890c-40a7c62e7615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:18.102960 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000a9f0-3b5f-40f6-890c-40a7c62e7615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:18.107010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000a9f0-3b5f-40f6-890c-40a7c62e7615 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:18.134108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1102/2215] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:18.135517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000a9f0-3b5f-40f6-890c-40a7c62e7615 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:18.138128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f000a9f0-3b5f-40f6-890c-40a7c62e7615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:18.188776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf90b4e09e94cc984fe36b03b08a0b6', 'request_id': 'req-2d7b025f-4bee-4840-a50a-aa9b105703d6'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:18.205378 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf90b4e09e94cc984fe36b03b08a0b6', 'request_id': 'req-2d7b025f-4bee-4840-a50a-aa9b105703d6'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:18.234430 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf90b4e09e94cc984fe36b03b08a0b6', 'request_id': 'req-2d7b025f-4bee-4840-a50a-aa9b105703d6'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:18.268151 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] The token cache is being invalidate because project 9bf90b4e09e94cc984fe36b03b08a0b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:18.269418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:18.271951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:18.276288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1114/2216] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:17 2022] DELETE /identity/v3/projects/9bf90b4e09e94cc984fe36b03b08a0b6 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:18.279023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:18.283210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:18.313317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000a9f0-3b5f-40f6-890c-40a7c62e7615 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:18.335927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:18.431171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:18.452771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f0e7d-39c3-44f0-b933-fb9ed2e0e773 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:18.454760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f0e7d-39c3-44f0-b933-fb9ed2e0e773 service swift] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:18.456556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f0e7d-39c3-44f0-b933-fb9ed2e0e773 service swift] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:18.457211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f0e7d-39c3-44f0-b933-fb9ed2e0e773 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:18.466679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0a6f0e7d-39c3-44f0-b933-fb9ed2e0e773 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:18.497965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b025f-4bee-4840-a50a-aa9b105703d6 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:18.515331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994ef8e-3fcf-42fd-ad07-a950703cfd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:18.521245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994ef8e-3fcf-42fd-ad07-a950703cfd5c admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:18.521662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994ef8e-3fcf-42fd-ad07-a950703cfd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:18.521849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994ef8e-3fcf-42fd-ad07-a950703cfd5c admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:18.524577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994ef8e-3fcf-42fd-ad07-a950703cfd5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:18.529169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994ef8e-3fcf-42fd-ad07-a950703cfd5c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:18.585125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1115/2217] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:18 2022] POST /identity/v3/users => generated 293 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:18.585125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994ef8e-3fcf-42fd-ad07-a950703cfd5c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:18.586718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9994ef8e-3fcf-42fd-ad07-a950703cfd5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:18.622629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994ef8e-3fcf-42fd-ad07-a950703cfd5c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:18.633113 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f0e7d-39c3-44f0-b933-fb9ed2e0e773 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:18.638857 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f0e7d-39c3-44f0-b933-fb9ed2e0e773 service swift] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:18.639967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2c56e-7d29-4b1e-9592-7e0e1efbe039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:18.643483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2c56e-7d29-4b1e-9592-7e0e1efbe039 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:18.643949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2c56e-7d29-4b1e-9592-7e0e1efbe039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:18.645331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2c56e-7d29-4b1e-9592-7e0e1efbe039 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:18.647194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2c56e-7d29-4b1e-9592-7e0e1efbe039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:18.654584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2c56e-7d29-4b1e-9592-7e0e1efbe039 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:18.663056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1103/2218] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:18:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:18:18.665835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501e662-9378-4b22-996e-7cea48532678 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:18.666347 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501e662-9378-4b22-996e-7cea48532678 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:18.666840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501e662-9378-4b22-996e-7cea48532678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:18.715980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-7501e662-9378-4b22-996e-7cea48532678 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:18.717964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7501e662-9378-4b22-996e-7cea48532678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:18.733234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1116/2219] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:18 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:18.738134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2c56e-7d29-4b1e-9592-7e0e1efbe039 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:18.855251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2c56e-7d29-4b1e-9592-7e0e1efbe039 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:18.862117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82505c-e7ea-4653-ae3e-e9a93c6d1fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:18.864072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82505c-e7ea-4653-ae3e-e9a93c6d1fad admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:18.864729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82505c-e7ea-4653-ae3e-e9a93c6d1fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:18.865252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82505c-e7ea-4653-ae3e-e9a93c6d1fad admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:18.866421 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82505c-e7ea-4653-ae3e-e9a93c6d1fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:18.869653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82505c-e7ea-4653-ae3e-e9a93c6d1fad admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:18.896337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1117/2220] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:18.897763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82505c-e7ea-4653-ae3e-e9a93c6d1fad admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:18.903707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1104/2221] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:18.905241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501e662-9378-4b22-996e-7cea48532678 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:18.914248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7501e662-9378-4b22-996e-7cea48532678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:18.929526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82505c-e7ea-4653-ae3e-e9a93c6d1fad admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:18.938602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501e662-9378-4b22-996e-7cea48532678 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:18.943831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87fdb0-5c09-424a-898b-32dc576051e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:18.945372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87fdb0-5c09-424a-898b-32dc576051e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:18.945920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87fdb0-5c09-424a-898b-32dc576051e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:18.946455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:18.946971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87fdb0-5c09-424a-898b-32dc576051e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:18.947132 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:18.947392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:18.947707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] PATH_INFO: `/v3/projects/958c56742a854fb88237f4ebc5ef5749` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:18.949022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87fdb0-5c09-424a-898b-32dc576051e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:18.949201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=958c56742a854fb88237f4ebc5ef5749)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:18.951320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:18.954300 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87fdb0-5c09-424a-898b-32dc576051e2 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:19.002817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1118/2222] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:18 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:19.011540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8209d-4fa1-4ed3-bf01-56adbdbff5bb None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:19.012052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8209d-4fa1-4ed3-bf01-56adbdbff5bb None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:19.012536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8209d-4fa1-4ed3-bf01-56adbdbff5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:19.048684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '958c56742a854fb88237f4ebc5ef5749', 'request_id': 'req-aa7a92a3-86b4-4daa-9264-db1765b2af82'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:19.058019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '958c56742a854fb88237f4ebc5ef5749', 'request_id': 'req-aa7a92a3-86b4-4daa-9264-db1765b2af82'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:19.058306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-efb8209d-4fa1-4ed3-bf01-56adbdbff5bb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:19.060442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-efb8209d-4fa1-4ed3-bf01-56adbdbff5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:19.070091 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '958c56742a854fb88237f4ebc5ef5749', 'request_id': 'req-aa7a92a3-86b4-4daa-9264-db1765b2af82'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:19.103221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] The token cache is being invalidate because project 958c56742a854fb88237f4ebc5ef5749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:19.103948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:19.105024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:19.107438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1105/2223] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:18 2022] DELETE /identity/v3/projects/958c56742a854fb88237f4ebc5ef5749 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:19.108543 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:19.110528 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:19.167841 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:19.220441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1119/2224] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:19.222289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8209d-4fa1-4ed3-bf01-56adbdbff5bb None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:19.224875 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-efb8209d-4fa1-4ed3-bf01-56adbdbff5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:19.237281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a92a3-86b4-4daa-9264-db1765b2af82 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:19.244083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d62d1c-eb72-4d0b-af23-32e078c6f11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:19.245131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d62d1c-eb72-4d0b-af23-32e078c6f11b admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:19.245606 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d62d1c-eb72-4d0b-af23-32e078c6f11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:19.246086 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d62d1c-eb72-4d0b-af23-32e078c6f11b admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:19.246370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8209d-4fa1-4ed3-bf01-56adbdbff5bb None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:19.247258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d62d1c-eb72-4d0b-af23-32e078c6f11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:19.250302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d62d1c-eb72-4d0b-af23-32e078c6f11b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:19.253541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f9fda-ff43-447f-b92f-7cc039744c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:19.254285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f9fda-ff43-447f-b92f-7cc039744c81 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:19.254573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f9fda-ff43-447f-b92f-7cc039744c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:19.255101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f9fda-ff43-447f-b92f-7cc039744c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:19.255905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f9fda-ff43-447f-b92f-7cc039744c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:19.257857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f9fda-ff43-447f-b92f-7cc039744c81 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:19.259871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:19.269608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1106/2225] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:19.274588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2871c-8515-467e-ae15-1daf78e29ac2 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:19.274862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2871c-8515-467e-ae15-1daf78e29ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:19.275128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2871c-8515-467e-ae15-1daf78e29ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:19.289119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1120/2226] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:19.292833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f9fda-ff43-447f-b92f-7cc039744c81 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:19.301909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-1ec2871c-8515-467e-ae15-1daf78e29ac2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:19.303373 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1ec2871c-8515-467e-ae15-1daf78e29ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:19.313377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1107/2227] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:19.314552 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2871c-8515-467e-ae15-1daf78e29ac2 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:19.316886 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f9fda-ff43-447f-b92f-7cc039744c81 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:19.327006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8435f-ed69-4c0f-9496-efd5c170f8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:19.327389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2871c-8515-467e-ae15-1daf78e29ac2 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:19.327867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8435f-ed69-4c0f-9496-efd5c170f8cb admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:19.328367 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8435f-ed69-4c0f-9496-efd5c170f8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:19.328766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8435f-ed69-4c0f-9496-efd5c170f8cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:19.329826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8435f-ed69-4c0f-9496-efd5c170f8cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:19.330280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8435f-ed69-4c0f-9496-efd5c170f8cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:19.332325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8435f-ed69-4c0f-9496-efd5c170f8cb admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:19.333675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620223c7-d0c6-42e6-88f3-b2011a92f0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:19.334626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620223c7-d0c6-42e6-88f3-b2011a92f0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:19.334937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620223c7-d0c6-42e6-88f3-b2011a92f0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:19.335221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620223c7-d0c6-42e6-88f3-b2011a92f0ae admin admin] PATH_INFO: `/v3/projects/41ecb37e0e2a41ad8bb21847629b94a1/users/01b13914e5db4520858e5c1692522b2c/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:19.356934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1121/2228] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:19.357578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620223c7-d0c6-42e6-88f3-b2011a92f0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ecb37e0e2a41ad8bb21847629b94a1, user_id=01b13914e5db4520858e5c1692522b2c, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:19.358710 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8435f-ed69-4c0f-9496-efd5c170f8cb admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:19.363259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620223c7-d0c6-42e6-88f3-b2011a92f0ae admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:19.372757 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8435f-ed69-4c0f-9496-efd5c170f8cb admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:19.386032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1108/2229] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:19 2022] PUT /identity/v3/projects/41ecb37e0e2a41ad8bb21847629b94a1/users/01b13914e5db4520858e5c1692522b2c/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:19.396607 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc94b7-2b7a-4de3-9d3e-2070bd3ba6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:19.397365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc94b7-2b7a-4de3-9d3e-2070bd3ba6d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:19.397677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc94b7-2b7a-4de3-9d3e-2070bd3ba6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:19.397831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc94b7-2b7a-4de3-9d3e-2070bd3ba6d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:19.398485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc94b7-2b7a-4de3-9d3e-2070bd3ba6d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:19.399387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84f194-dc5d-4fd1-961d-c7de6229f37e None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:19.399670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84f194-dc5d-4fd1-961d-c7de6229f37e None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:19.399944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84f194-dc5d-4fd1-961d-c7de6229f37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:19.400197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc94b7-2b7a-4de3-9d3e-2070bd3ba6d4 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:19.427861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8a84f194-dc5d-4fd1-961d-c7de6229f37e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:19.429362 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8a84f194-dc5d-4fd1-961d-c7de6229f37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:19.439153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1122/2230] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:19.440900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc94b7-2b7a-4de3-9d3e-2070bd3ba6d4 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:19.470269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc94b7-2b7a-4de3-9d3e-2070bd3ba6d4 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:19.555144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1109/2231] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:19.556617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae6183-d64e-4b4f-af63-116d4d9db9de None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:19.556900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae6183-d64e-4b4f-af63-116d4d9db9de None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:19.557175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae6183-d64e-4b4f-af63-116d4d9db9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:19.600273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-0bae6183-d64e-4b4f-af63-116d4d9db9de None None] MFA Rules not processed for user `01b13914e5db4520858e5c1692522b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:19.602105 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0bae6183-d64e-4b4f-af63-116d4d9db9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:19.607437 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11cc5a-a5dd-42cb-a179-d3741d19f896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:19.610087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11cc5a-a5dd-42cb-a179-d3741d19f896 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:19.610617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11cc5a-a5dd-42cb-a179-d3741d19f896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:19.611633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11cc5a-a5dd-42cb-a179-d3741d19f896 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:19.613550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11cc5a-a5dd-42cb-a179-d3741d19f896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:19.617264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11cc5a-a5dd-42cb-a179-d3741d19f896 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:19.671323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1123/2232] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:19 2022] POST /identity/v3/projects => generated 371 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:19.673472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11cc5a-a5dd-42cb-a179-d3741d19f896 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:19.759342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1110/2233] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:19.762577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae6183-d64e-4b4f-af63-116d4d9db9de None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:19.791631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae6183-d64e-4b4f-af63-116d4d9db9de None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:19.825635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11cc5a-a5dd-42cb-a179-d3741d19f896 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:19.830548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:19.836522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da0ee4-867e-4d74-b4b4-158dc64b3ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:19.838121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da0ee4-867e-4d74-b4b4-158dc64b3ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:19.838412 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da0ee4-867e-4d74-b4b4-158dc64b3ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:19.839037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da0ee4-867e-4d74-b4b4-158dc64b3ddf admin admin] PATH_INFO: `/v3/projects/4de7d1725db046779e53b8ef175bb49d/users/bcc5e269815b4a3b81cdaf7f447a518f/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:19.864223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da0ee4-867e-4d74-b4b4-158dc64b3ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de7d1725db046779e53b8ef175bb49d, user_id=bcc5e269815b4a3b81cdaf7f447a518f, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:19.869249 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da0ee4-867e-4d74-b4b4-158dc64b3ddf admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:19.895665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1124/2234] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:19 2022] PUT /identity/v3/projects/4de7d1725db046779e53b8ef175bb49d/users/bcc5e269815b4a3b81cdaf7f447a518f/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:19.896276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da0ee4-867e-4d74-b4b4-158dc64b3ddf admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:19.919305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda9cda-defa-4428-84ff-3309162cd15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:19.920409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda9cda-defa-4428-84ff-3309162cd15e admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:19.920788 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda9cda-defa-4428-84ff-3309162cd15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:19.921157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda9cda-defa-4428-84ff-3309162cd15e admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:19.925138 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda9cda-defa-4428-84ff-3309162cd15e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:19.926447 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda9cda-defa-4428-84ff-3309162cd15e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:19.928532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda9cda-defa-4428-84ff-3309162cd15e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:19.946520 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1111/2235] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:19.947306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda9cda-defa-4428-84ff-3309162cd15e admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:19.962823 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda9cda-defa-4428-84ff-3309162cd15e admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:19.999701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:20.028855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da0ee4-867e-4d74-b4b4-158dc64b3ddf admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.037189 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301f6d1-73d0-4106-923c-5bc70774b7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.037829 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f6d1-73d0-4106-923c-5bc70774b7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.038106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f6d1-73d0-4106-923c-5bc70774b7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.038412 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f6d1-73d0-4106-923c-5bc70774b7d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.039037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301f6d1-73d0-4106-923c-5bc70774b7d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.041540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301f6d1-73d0-4106-923c-5bc70774b7d1 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.067128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df97bc1-7d03-4a71-b846-53d616e878fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.067844 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df97bc1-7d03-4a71-b846-53d616e878fa admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.068176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df97bc1-7d03-4a71-b846-53d616e878fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.068448 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df97bc1-7d03-4a71-b846-53d616e878fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.069158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df97bc1-7d03-4a71-b846-53d616e878fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.070896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1125/2236] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:19 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:20.075371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df97bc1-7d03-4a71-b846-53d616e878fa admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.075518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e550969-aa48-4fcd-9d49-2e0ff53029a8 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.075518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e550969-aa48-4fcd-9d49-2e0ff53029a8 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.075518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e550969-aa48-4fcd-9d49-2e0ff53029a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.124413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1112/2237] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:19 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:20.126232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df97bc1-7d03-4a71-b846-53d616e878fa admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.133866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-5e550969-aa48-4fcd-9d49-2e0ff53029a8 None None] MFA Rules not processed for user `bcc5e269815b4a3b81cdaf7f447a518f`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:20.141857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5e550969-aa48-4fcd-9d49-2e0ff53029a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:20.226570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df97bc1-7d03-4a71-b846-53d616e878fa admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.234572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b60d0-6459-4206-a38d-23bb809f1cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.235218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b60d0-6459-4206-a38d-23bb809f1cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.235491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b60d0-6459-4206-a38d-23bb809f1cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.235748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b60d0-6459-4206-a38d-23bb809f1cb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.237325 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b60d0-6459-4206-a38d-23bb809f1cb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.239375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b60d0-6459-4206-a38d-23bb809f1cb7 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.274410 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1113/2238] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:20 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:20.275841 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1daca-0ecc-49fb-b4d4-162931fb0782 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.276218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1daca-0ecc-49fb-b4d4-162931fb0782 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.276490 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1daca-0ecc-49fb-b4d4-162931fb0782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.307203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-66c1daca-0ecc-49fb-b4d4-162931fb0782 None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:20.308894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1126/2239] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:20.310060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e550969-aa48-4fcd-9d49-2e0ff53029a8 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.313451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-66c1daca-0ecc-49fb-b4d4-162931fb0782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:20.333031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e550969-aa48-4fcd-9d49-2e0ff53029a8 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.353891 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:20.439910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1114/2240] 10.222.0.44 () {58 vars in 967 bytes} [Tue Aug 30 01:18:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:20.441297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1daca-0ecc-49fb-b4d4-162931fb0782 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.448156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4764f-d285-4152-9b84-19e2c429c261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.448966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4764f-d285-4152-9b84-19e2c429c261 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.449304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4764f-d285-4152-9b84-19e2c429c261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.449578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4764f-d285-4152-9b84-19e2c429c261 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.450636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4764f-d285-4152-9b84-19e2c429c261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.452491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4764f-d285-4152-9b84-19e2c429c261 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.466973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1127/2241] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:20.468214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4764f-d285-4152-9b84-19e2c429c261 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.484586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4764f-d285-4152-9b84-19e2c429c261 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.491919 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeeee4-929a-4921-b3dd-ba9ad3a1be3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.493260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeeee4-929a-4921-b3dd-ba9ad3a1be3d admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.493955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeeee4-929a-4921-b3dd-ba9ad3a1be3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.494384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeeee4-929a-4921-b3dd-ba9ad3a1be3d admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.495716 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eeeee4-929a-4921-b3dd-ba9ad3a1be3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.499355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eeeee4-929a-4921-b3dd-ba9ad3a1be3d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.554851 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1daca-0ecc-49fb-b4d4-162931fb0782 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.560231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1128/2242] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:20 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:20.560734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66424b1d-0890-4835-abc4-889e3e681e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.561133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeeee4-929a-4921-b3dd-ba9ad3a1be3d admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.561380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66424b1d-0890-4835-abc4-889e3e681e01 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.562051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66424b1d-0890-4835-abc4-889e3e681e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.562051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66424b1d-0890-4835-abc4-889e3e681e01 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.562935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66424b1d-0890-4835-abc4-889e3e681e01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.564904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66424b1d-0890-4835-abc4-889e3e681e01 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.565235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-03eeeee4-929a-4921-b3dd-ba9ad3a1be3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:20.605954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1115/2243] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:20 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:20.607770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66424b1d-0890-4835-abc4-889e3e681e01 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.611065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-66424b1d-0890-4835-abc4-889e3e681e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:20.637365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:20.661925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeeee4-929a-4921-b3dd-ba9ad3a1be3d admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.667306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1d638-c170-43ec-b6c7-d87ff6a3f0e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.668834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1d638-c170-43ec-b6c7-d87ff6a3f0e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.670097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1d638-c170-43ec-b6c7-d87ff6a3f0e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.670387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1d638-c170-43ec-b6c7-d87ff6a3f0e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.720673 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66424b1d-0890-4835-abc4-889e3e681e01 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.727403 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655ece0-9c76-4221-b584-37a5c54c1ae0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.729037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655ece0-9c76-4221-b584-37a5c54c1ae0 service swift] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.729581 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655ece0-9c76-4221-b584-37a5c54c1ae0 service swift] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.729994 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655ece0-9c76-4221-b584-37a5c54c1ae0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.740422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655ece0-9c76-4221-b584-37a5c54c1ae0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.742131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655ece0-9c76-4221-b584-37a5c54c1ae0 service swift] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.765092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1116/2244] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:18:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:18:20.766690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655ece0-9c76-4221-b584-37a5c54c1ae0 service swift] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.780535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1d638-c170-43ec-b6c7-d87ff6a3f0e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.783293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1d638-c170-43ec-b6c7-d87ff6a3f0e5 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.791211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655ece0-9c76-4221-b584-37a5c54c1ae0 service swift] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.795379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1129/2245] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:20.797896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1d638-c170-43ec-b6c7-d87ff6a3f0e5 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.806055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c158194-8b2b-4bd9-8b79-acc3f7e3a8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.806793 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c158194-8b2b-4bd9-8b79-acc3f7e3a8dd admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.807060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c158194-8b2b-4bd9-8b79-acc3f7e3a8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.807327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c158194-8b2b-4bd9-8b79-acc3f7e3a8dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.807980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c158194-8b2b-4bd9-8b79-acc3f7e3a8dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.809628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c158194-8b2b-4bd9-8b79-acc3f7e3a8dd admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.812734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1d638-c170-43ec-b6c7-d87ff6a3f0e5 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.830630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf675a3-3bf2-4964-83ac-87fa9191d8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.831667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf675a3-3bf2-4964-83ac-87fa9191d8cc admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.831947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf675a3-3bf2-4964-83ac-87fa9191d8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.832239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf675a3-3bf2-4964-83ac-87fa9191d8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.835945 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf675a3-3bf2-4964-83ac-87fa9191d8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.836170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1117/2246] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:20.836170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c158194-8b2b-4bd9-8b79-acc3f7e3a8dd admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.836345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf675a3-3bf2-4964-83ac-87fa9191d8cc admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.847736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c158194-8b2b-4bd9-8b79-acc3f7e3a8dd admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.860254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1130/2247] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:20.860254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf675a3-3bf2-4964-83ac-87fa9191d8cc admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.868616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e832925-1d29-42c0-863a-b9b71eb761c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.869397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e832925-1d29-42c0-863a-b9b71eb761c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.869667 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e832925-1d29-42c0-863a-b9b71eb761c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.869934 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e832925-1d29-42c0-863a-b9b71eb761c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.871286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e832925-1d29-42c0-863a-b9b71eb761c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.872728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e832925-1d29-42c0-863a-b9b71eb761c6 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.884246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf675a3-3bf2-4964-83ac-87fa9191d8cc admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.892607 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1118/2248] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:20.893531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e832925-1d29-42c0-863a-b9b71eb761c6 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.894579 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01879447-42fb-41d0-9d9b-4a21bc7bad46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.895261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01879447-42fb-41d0-9d9b-4a21bc7bad46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.895537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01879447-42fb-41d0-9d9b-4a21bc7bad46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.895805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01879447-42fb-41d0-9d9b-4a21bc7bad46 admin admin] PATH_INFO: `/v3/projects/0c65a92eef9744e7ba23b4bb06dd1c81/users/60581a263dd84eec82e30155f4156dee/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.907593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e832925-1d29-42c0-863a-b9b71eb761c6 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.913080 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcb412-3feb-4877-abf4-4800935903c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:20.913696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcb412-3feb-4877-abf4-4800935903c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:20.913969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcb412-3feb-4877-abf4-4800935903c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:20.914242 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcb412-3feb-4877-abf4-4800935903c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:20.914876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdcb412-3feb-4877-abf4-4800935903c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.919040 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdcb412-3feb-4877-abf4-4800935903c2 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.929556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01879447-42fb-41d0-9d9b-4a21bc7bad46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c65a92eef9744e7ba23b4bb06dd1c81, user_id=60581a263dd84eec82e30155f4156dee, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:20.933568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01879447-42fb-41d0-9d9b-4a21bc7bad46 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:20.939426 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1119/2249] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:20.939426 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcb412-3feb-4877-abf4-4800935903c2 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.952032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcb412-3feb-4877-abf4-4800935903c2 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:20.961117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1131/2250] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:20 2022] PUT /identity/v3/projects/0c65a92eef9744e7ba23b4bb06dd1c81/users/60581a263dd84eec82e30155f4156dee/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:20.962022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01879447-42fb-41d0-9d9b-4a21bc7bad46 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:20.986219 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:21.089191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc0dfb-7ced-4b76-8cac-030a0bc94946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:21.090058 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc0dfb-7ced-4b76-8cac-030a0bc94946 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:21.090342 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc0dfb-7ced-4b76-8cac-030a0bc94946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:21.090761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc0dfb-7ced-4b76-8cac-030a0bc94946 admin admin] PATH_INFO: `/v3/projects/4f63c452341f48c5ab62568c94e3cdfe/users/50527af40b2f466097bb9bdc4a652e3d/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:21.092169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01879447-42fb-41d0-9d9b-4a21bc7bad46 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:21.099687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409dde2-9d91-494a-8446-c76795fe2b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:21.100464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409dde2-9d91-494a-8446-c76795fe2b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:21.100750 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409dde2-9d91-494a-8446-c76795fe2b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:21.101024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409dde2-9d91-494a-8446-c76795fe2b34 admin admin] PATH_INFO: `/v3/projects/7dd8aa7f0b654dd1bd05af1ea56e6aee/users/40b34b8d24b540f3a72da2b1eb4c37de/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:21.117821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc0dfb-7ced-4b76-8cac-030a0bc94946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f63c452341f48c5ab62568c94e3cdfe, user_id=50527af40b2f466097bb9bdc4a652e3d, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:21.122878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc0dfb-7ced-4b76-8cac-030a0bc94946 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:21.127408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409dde2-9d91-494a-8446-c76795fe2b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dd8aa7f0b654dd1bd05af1ea56e6aee, user_id=40b34b8d24b540f3a72da2b1eb4c37de, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:21.131183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409dde2-9d91-494a-8446-c76795fe2b34 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:21.150533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1120/2251] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:20 2022] PUT /identity/v3/projects/4f63c452341f48c5ab62568c94e3cdfe/users/50527af40b2f466097bb9bdc4a652e3d/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:21.156174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413e7db-ed39-4021-9bab-8d2bbf7e285e None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:21.156646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413e7db-ed39-4021-9bab-8d2bbf7e285e None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:21.157205 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413e7db-ed39-4021-9bab-8d2bbf7e285e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:21.163060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1132/2252] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:20 2022] PUT /identity/v3/projects/7dd8aa7f0b654dd1bd05af1ea56e6aee/users/40b34b8d24b540f3a72da2b1eb4c37de/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:21.164377 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409dde2-9d91-494a-8446-c76795fe2b34 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:21.210792 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-1413e7db-ed39-4021-9bab-8d2bbf7e285e None None] MFA Rules not processed for user `60581a263dd84eec82e30155f4156dee`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:21.212281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1413e7db-ed39-4021-9bab-8d2bbf7e285e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:21.225137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:21.263068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409dde2-9d91-494a-8446-c76795fe2b34 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:21.283305 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77862d48-0eb7-4034-8789-4c3d8653e554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:21.284303 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77862d48-0eb7-4034-8789-4c3d8653e554 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:21.284981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77862d48-0eb7-4034-8789-4c3d8653e554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:21.285403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77862d48-0eb7-4034-8789-4c3d8653e554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:21.286608 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77862d48-0eb7-4034-8789-4c3d8653e554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:21.291121 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77862d48-0eb7-4034-8789-4c3d8653e554 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:21.315578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1133/2253] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:21.317803 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cf9b6-d5cd-489f-a44e-f11cb4f0b62d None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:21.318050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cf9b6-d5cd-489f-a44e-f11cb4f0b62d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:21.318354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cf9b6-d5cd-489f-a44e-f11cb4f0b62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:21.361074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1121/2254] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:21.366181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3622a33-c9db-4e64-b8b2-63f32b6f8741 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:21.368657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3622a33-c9db-4e64-b8b2-63f32b6f8741 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:21.369750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3622a33-c9db-4e64-b8b2-63f32b6f8741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:21.380593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ec7cf9b6-d5cd-489f-a44e-f11cb4f0b62d None None] MFA Rules not processed for user `40b34b8d24b540f3a72da2b1eb4c37de`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:21.384541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ec7cf9b6-d5cd-489f-a44e-f11cb4f0b62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:21.429077 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-b3622a33-c9db-4e64-b8b2-63f32b6f8741 None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:21.432583 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b3622a33-c9db-4e64-b8b2-63f32b6f8741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:21.574199 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1122/2255] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:18:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:21.574797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3622a33-c9db-4e64-b8b2-63f32b6f8741 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:21.589629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3622a33-c9db-4e64-b8b2-63f32b6f8741 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:21.595579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3ea5f-8ea6-4cbc-b480-adcedf556171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:21.596410 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3ea5f-8ea6-4cbc-b480-adcedf556171 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:21.596887 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3ea5f-8ea6-4cbc-b480-adcedf556171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:21.597378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3ea5f-8ea6-4cbc-b480-adcedf556171 admin admin] PATH_INFO: `/v3/projects/4f63c452341f48c5ab62568c94e3cdfe/users/50527af40b2f466097bb9bdc4a652e3d/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:21.599870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1134/2256] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:21.601012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3ea5f-8ea6-4cbc-b480-adcedf556171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f63c452341f48c5ab62568c94e3cdfe, user_id=50527af40b2f466097bb9bdc4a652e3d, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:21.603790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3ea5f-8ea6-4cbc-b480-adcedf556171 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:21.605427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cf9b6-d5cd-489f-a44e-f11cb4f0b62d None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:21.617611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1123/2257] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:21 2022] PUT /identity/v3/projects/4f63c452341f48c5ab62568c94e3cdfe/users/50527af40b2f466097bb9bdc4a652e3d/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:21.618672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3ea5f-8ea6-4cbc-b480-adcedf556171 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:21.623815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cf9b6-d5cd-489f-a44e-f11cb4f0b62d None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:21.652631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:21.741487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3ea5f-8ea6-4cbc-b480-adcedf556171 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:21.747864 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c1929-57b2-46d6-829b-121111fa47af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:21.748115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092d52e-9d6e-4f7a-85a5-92c12de2dd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:21.749262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092d52e-9d6e-4f7a-85a5-92c12de2dd0c admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:21.749417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c1929-57b2-46d6-829b-121111fa47af service swift] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:21.749827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c1929-57b2-46d6-829b-121111fa47af service swift] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:21.750489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c1929-57b2-46d6-829b-121111fa47af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:21.750725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092d52e-9d6e-4f7a-85a5-92c12de2dd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:21.751120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092d52e-9d6e-4f7a-85a5-92c12de2dd0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:21.752020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092d52e-9d6e-4f7a-85a5-92c12de2dd0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:21.754599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092d52e-9d6e-4f7a-85a5-92c12de2dd0c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:21.798972 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1135/2258] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:21 2022] POST /identity/v3/projects => generated 373 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:21.801157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092d52e-9d6e-4f7a-85a5-92c12de2dd0c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:21.841065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c1929-57b2-46d6-829b-121111fa47af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:21.842738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c1929-57b2-46d6-829b-121111fa47af service swift] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:21.864712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1124/2259] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:18:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:18:21.866637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce6c11-b75f-43c3-b4a8-1f6cb2efc1e3 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:21.867029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce6c11-b75f-43c3-b4a8-1f6cb2efc1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:21.867381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce6c11-b75f-43c3-b4a8-1f6cb2efc1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:21.873688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:21.895469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092d52e-9d6e-4f7a-85a5-92c12de2dd0c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:21.902473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a179b7-86ef-457d-ad62-3e1f096075f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:21.903535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a179b7-86ef-457d-ad62-3e1f096075f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:21.904168 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a179b7-86ef-457d-ad62-3e1f096075f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:21.904783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a179b7-86ef-457d-ad62-3e1f096075f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:21.905627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a179b7-86ef-457d-ad62-3e1f096075f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:21.908305 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a179b7-86ef-457d-ad62-3e1f096075f3 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:21.919574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8fce6c11-b75f-43c3-b4a8-1f6cb2efc1e3 None None] MFA Rules not processed for user `50527af40b2f466097bb9bdc4a652e3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:21.920996 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8fce6c11-b75f-43c3-b4a8-1f6cb2efc1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:21.948859 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1136/2260] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:21 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:21.949877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a179b7-86ef-457d-ad62-3e1f096075f3 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:22.061374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1125/2261] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:22.063570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce6c11-b75f-43c3-b4a8-1f6cb2efc1e3 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:22.076954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce6c11-b75f-43c3-b4a8-1f6cb2efc1e3 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:22.115354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a179b7-86ef-457d-ad62-3e1f096075f3 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:22.120877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdbb0a-c80f-44e1-a692-5ef9d34d57c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:22.121637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdbb0a-c80f-44e1-a692-5ef9d34d57c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:22.121920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdbb0a-c80f-44e1-a692-5ef9d34d57c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:22.122188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdbb0a-c80f-44e1-a692-5ef9d34d57c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:22.122834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cdbb0a-c80f-44e1-a692-5ef9d34d57c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:22.125539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cdbb0a-c80f-44e1-a692-5ef9d34d57c0 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:22.150202 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:22.167958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ede9d-2bb9-40ba-9fbd-d9970c294b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:22.168780 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ede9d-2bb9-40ba-9fbd-d9970c294b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:22.169158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ede9d-2bb9-40ba-9fbd-d9970c294b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:22.169568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ede9d-2bb9-40ba-9fbd-d9970c294b46 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:22.170384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ede9d-2bb9-40ba-9fbd-d9970c294b46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:22.173200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ede9d-2bb9-40ba-9fbd-d9970c294b46 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:22.178630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1137/2262] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:21 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:22.178630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdbb0a-c80f-44e1-a692-5ef9d34d57c0 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:22.180545 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a9cdbb0a-c80f-44e1-a692-5ef9d34d57c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:22.226239 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1126/2263] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:22 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:22.227101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ede9d-2bb9-40ba-9fbd-d9970c294b46 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:22.241980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ede9d-2bb9-40ba-9fbd-d9970c294b46 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:22.253577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdae83-1860-471e-9e0c-7c7f9dc31b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:22.256054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdae83-1860-471e-9e0c-7c7f9dc31b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:22.256489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdae83-1860-471e-9e0c-7c7f9dc31b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:22.256921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdae83-1860-471e-9e0c-7c7f9dc31b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:22.257988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cdae83-1860-471e-9e0c-7c7f9dc31b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:22.260398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cdae83-1860-471e-9e0c-7c7f9dc31b4f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:22.277664 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1127/2264] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:22.279453 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdae83-1860-471e-9e0c-7c7f9dc31b4f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:22.302277 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdae83-1860-471e-9e0c-7c7f9dc31b4f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:22.311493 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7eca8-66d1-4df3-bf66-32251a2f9f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:22.312980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7eca8-66d1-4df3-bf66-32251a2f9f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:22.313262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7eca8-66d1-4df3-bf66-32251a2f9f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:22.313616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7eca8-66d1-4df3-bf66-32251a2f9f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:22.314672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7eca8-66d1-4df3-bf66-32251a2f9f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:22.317027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7eca8-66d1-4df3-bf66-32251a2f9f1c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:22.327815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdbb0a-c80f-44e1-a692-5ef9d34d57c0 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:22.334657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9653172-0db2-436c-9c3d-fbc5b238d52a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:22.335471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9653172-0db2-436c-9c3d-fbc5b238d52a service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:22.335862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9653172-0db2-436c-9c3d-fbc5b238d52a service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:22.336292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9653172-0db2-436c-9c3d-fbc5b238d52a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:22.344818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1128/2265] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:22.346139 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7eca8-66d1-4df3-bf66-32251a2f9f1c admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:22.368315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7eca8-66d1-4df3-bf66-32251a2f9f1c admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:22.378274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4758c5e-9b0f-4943-92f8-b4cefba68c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:22.379148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4758c5e-9b0f-4943-92f8-b4cefba68c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:22.379418 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4758c5e-9b0f-4943-92f8-b4cefba68c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:22.379692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4758c5e-9b0f-4943-92f8-b4cefba68c05 admin admin] PATH_INFO: `/v3/projects/ebb248869d024171a9126701f6f27aea/users/4133f0859be546269c2b5b1a5418d8c4/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:22.421627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4758c5e-9b0f-4943-92f8-b4cefba68c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb248869d024171a9126701f6f27aea, user_id=4133f0859be546269c2b5b1a5418d8c4, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:22.435831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4758c5e-9b0f-4943-92f8-b4cefba68c05 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:22.457535 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9653172-0db2-436c-9c3d-fbc5b238d52a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:22.461657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9653172-0db2-436c-9c3d-fbc5b238d52a service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:22.492456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1129/2266] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:22 2022] PUT /identity/v3/projects/ebb248869d024171a9126701f6f27aea/users/4133f0859be546269c2b5b1a5418d8c4/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:22.493599 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4758c5e-9b0f-4943-92f8-b4cefba68c05 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:22.518402 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1138/2267] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:22.520031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9653172-0db2-436c-9c3d-fbc5b238d52a service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:22.603255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9653172-0db2-436c-9c3d-fbc5b238d52a service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:22.643156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4758c5e-9b0f-4943-92f8-b4cefba68c05 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:22.679188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:22.684021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64773aec-91d4-4c8e-b019-fffb4714b5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:22.700513 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64773aec-91d4-4c8e-b019-fffb4714b5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:22.700822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64773aec-91d4-4c8e-b019-fffb4714b5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:22.717964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64773aec-91d4-4c8e-b019-fffb4714b5ad admin admin] PATH_INFO: `/v3/projects/d2813420736a4b81915e556a1c2a266b/users/4f7f03e1d94d405da53513b0061335fb/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:22.757912 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64773aec-91d4-4c8e-b019-fffb4714b5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2813420736a4b81915e556a1c2a266b, user_id=4f7f03e1d94d405da53513b0061335fb, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:22.761208 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64773aec-91d4-4c8e-b019-fffb4714b5ad admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:22.800245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1130/2268] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:22 2022] PUT /identity/v3/projects/d2813420736a4b81915e556a1c2a266b/users/4f7f03e1d94d405da53513b0061335fb/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:22.802769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64773aec-91d4-4c8e-b019-fffb4714b5ad admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:22.938041 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb2e17-bcf4-4244-bc7f-6be6a72cb07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:22.939078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb2e17-bcf4-4244-bc7f-6be6a72cb07e admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:22.939547 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb2e17-bcf4-4244-bc7f-6be6a72cb07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:22.940026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb2e17-bcf4-4244-bc7f-6be6a72cb07e admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:22.950074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb2e17-bcf4-4244-bc7f-6be6a72cb07e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:22.955500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb2e17-bcf4-4244-bc7f-6be6a72cb07e admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:22.994974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1139/2269] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:23.003323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb2e17-bcf4-4244-bc7f-6be6a72cb07e admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:23.030164 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64773aec-91d4-4c8e-b019-fffb4714b5ad admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:23.055289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb2e17-bcf4-4244-bc7f-6be6a72cb07e admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:23.057644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935ec5a-8283-41fa-b307-cc1377572015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:23.058740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935ec5a-8283-41fa-b307-cc1377572015 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:23.059238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935ec5a-8283-41fa-b307-cc1377572015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:23.059742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935ec5a-8283-41fa-b307-cc1377572015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:23.061185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935ec5a-8283-41fa-b307-cc1377572015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:23.067062 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935ec5a-8283-41fa-b307-cc1377572015 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:23.076484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abd4ce-295d-41d0-9054-4ebb6ce897fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:23.077645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abd4ce-295d-41d0-9054-4ebb6ce897fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:23.079200 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abd4ce-295d-41d0-9054-4ebb6ce897fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:23.079573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abd4ce-295d-41d0-9054-4ebb6ce897fa admin admin] PATH_INFO: `/v3/projects/ebb248869d024171a9126701f6f27aea/users/4133f0859be546269c2b5b1a5418d8c4/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:23.090968 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94abd4ce-295d-41d0-9054-4ebb6ce897fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb248869d024171a9126701f6f27aea, user_id=4133f0859be546269c2b5b1a5418d8c4, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:23.099489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94abd4ce-295d-41d0-9054-4ebb6ce897fa admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:23.111744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1131/2270] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:23.114887 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00190e-78e1-4646-bb3f-36bf065d5a8e None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:23.116199 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00190e-78e1-4646-bb3f-36bf065d5a8e None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:23.117275 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00190e-78e1-4646-bb3f-36bf065d5a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:23.128056 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1140/2271] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:22 2022] PUT /identity/v3/projects/ebb248869d024171a9126701f6f27aea/users/4133f0859be546269c2b5b1a5418d8c4/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:23.128403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abd4ce-295d-41d0-9054-4ebb6ce897fa admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:23.168326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-db00190e-78e1-4646-bb3f-36bf065d5a8e None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:23.169713 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-db00190e-78e1-4646-bb3f-36bf065d5a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:23.176266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:23.324462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1132/2272] 10.222.0.44 () {58 vars in 967 bytes} [Tue Aug 30 01:18:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:23.327211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563b3b1-acfb-4698-90a3-a338265ad9a1 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:23.327740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563b3b1-acfb-4698-90a3-a338265ad9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:23.328291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563b3b1-acfb-4698-90a3-a338265ad9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:23.338450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abd4ce-295d-41d0-9054-4ebb6ce897fa admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:23.356365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6d666-8eb9-4a57-aff5-e814b7024db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:23.357141 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6d666-8eb9-4a57-aff5-e814b7024db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:23.357455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6d666-8eb9-4a57-aff5-e814b7024db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:23.357737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6d666-8eb9-4a57-aff5-e814b7024db6 admin admin] PATH_INFO: `/v3/projects/d2813420736a4b81915e556a1c2a266b/users/4f7f03e1d94d405da53513b0061335fb/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:23.362471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6d666-8eb9-4a57-aff5-e814b7024db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2813420736a4b81915e556a1c2a266b, user_id=4f7f03e1d94d405da53513b0061335fb, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:23.366416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6d666-8eb9-4a57-aff5-e814b7024db6 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:23.406498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1141/2273] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:23 2022] PUT /identity/v3/projects/d2813420736a4b81915e556a1c2a266b/users/4f7f03e1d94d405da53513b0061335fb/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:23.406498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6d666-8eb9-4a57-aff5-e814b7024db6 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:23.426526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8563b3b1-acfb-4698-90a3-a338265ad9a1 None None] MFA Rules not processed for user `4133f0859be546269c2b5b1a5418d8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:23.427785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8563b3b1-acfb-4698-90a3-a338265ad9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:23.558925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6d666-8eb9-4a57-aff5-e814b7024db6 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:23.572198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43769b-5e94-4c99-8588-ff70840f57ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:23.573010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43769b-5e94-4c99-8588-ff70840f57ba service swift] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:23.573359 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43769b-5e94-4c99-8588-ff70840f57ba service swift] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:23.573855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43769b-5e94-4c99-8588-ff70840f57ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:23.628838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1133/2274] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:23.637608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b773e6-d1e5-47db-9370-288c1e892c81 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:23.638071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b773e6-d1e5-47db-9370-288c1e892c81 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:23.638428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b773e6-d1e5-47db-9370-288c1e892c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:23.688216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:23.717030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-a8b773e6-d1e5-47db-9370-288c1e892c81 None None] MFA Rules not processed for user `4f7f03e1d94d405da53513b0061335fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:23.718433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a8b773e6-d1e5-47db-9370-288c1e892c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:23.749103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43769b-5e94-4c99-8588-ff70840f57ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:23.751312 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43769b-5e94-4c99-8588-ff70840f57ba service swift] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:23.762624 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1142/2275] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:18:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:18:23.763733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43769b-5e94-4c99-8588-ff70840f57ba service swift] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:23.876930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43769b-5e94-4c99-8588-ff70840f57ba service swift] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:23.882671 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c33f9f-d49e-4dbf-93c8-785ffaa61213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:23.883789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c33f9f-d49e-4dbf-93c8-785ffaa61213 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:23.884356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c33f9f-d49e-4dbf-93c8-785ffaa61213 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:23.885132 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c33f9f-d49e-4dbf-93c8-785ffaa61213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:23.903242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c33f9f-d49e-4dbf-93c8-785ffaa61213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:23.905527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c33f9f-d49e-4dbf-93c8-785ffaa61213 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:23.913881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1134/2276] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:23.927249 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1143/2277] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:23.979303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b773e6-d1e5-47db-9370-288c1e892c81 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:23.991287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b773e6-d1e5-47db-9370-288c1e892c81 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:23.995431 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e685df-d640-4ee4-9792-c2bd45f29155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:23.996190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e685df-d640-4ee4-9792-c2bd45f29155 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:23.996570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e685df-d640-4ee4-9792-c2bd45f29155 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:23.997505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e685df-d640-4ee4-9792-c2bd45f29155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:24.009625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c33f9f-d49e-4dbf-93c8-785ffaa61213 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:24.044789 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:24.112739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e685df-d640-4ee4-9792-c2bd45f29155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:24.114593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e685df-d640-4ee4-9792-c2bd45f29155 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:24.126060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1135/2278] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:24.141193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c33f9f-d49e-4dbf-93c8-785ffaa61213 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:24.150079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:24.150863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:24.151291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:24.151520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] PATH_INFO: `/v3/users/01b13914e5db4520858e5c1692522b2c` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:24.152993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01b13914e5db4520858e5c1692522b2c)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:24.155194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:24.228522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01b13914e5db4520858e5c1692522b2c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:24.254423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01b13914e5db4520858e5c1692522b2c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:24.265819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01b13914e5db4520858e5c1692522b2c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:24.270736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:24.295948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01b13914e5db4520858e5c1692522b2c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:24.297270 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01b13914e5db4520858e5c1692522b2c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:24.309321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1144/2279] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:24 2022] DELETE /identity/v3/users/01b13914e5db4520858e5c1692522b2c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:24.317776 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaa8fc-9406-40ad-b9bc-6058b8ac55ba None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:24.318198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaa8fc-9406-40ad-b9bc-6058b8ac55ba None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:24.318535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaa8fc-9406-40ad-b9bc-6058b8ac55ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:24.345805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-edfaa8fc-9406-40ad-b9bc-6058b8ac55ba None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:24.347330 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-edfaa8fc-9406-40ad-b9bc-6058b8ac55ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:24.503393 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1136/2280] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:24.514100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:24.517039 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:24.598931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfaa8fc-9406-40ad-b9bc-6058b8ac55ba None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:24.601434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-edfaa8fc-9406-40ad-b9bc-6058b8ac55ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:24.629963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfaa8fc-9406-40ad-b9bc-6058b8ac55ba None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:24.651380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:24.654842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:24.656986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:24.657892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] PATH_INFO: `/v3/users/bcc5e269815b4a3b81cdaf7f447a518f` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:24.668656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc5e269815b4a3b81cdaf7f447a518f)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:24.673417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:24.690392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1d672-7875-4e7c-a46a-15faa5ade3e0 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:24.696887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61be9d-318e-4876-83a1-154d7effc5ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:24.699102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61be9d-318e-4876-83a1-154d7effc5ae service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:24.699578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61be9d-318e-4876-83a1-154d7effc5ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:24.700064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61be9d-318e-4876-83a1-154d7effc5ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:24.757531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61be9d-318e-4876-83a1-154d7effc5ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:24.762677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61be9d-318e-4876-83a1-154d7effc5ae service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:24.833595 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1145/2281] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:24.850074 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc5e269815b4a3b81cdaf7f447a518f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:24.879813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc5e269815b4a3b81cdaf7f447a518f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:24.880711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc5e269815b4a3b81cdaf7f447a518f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:24.881782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc5e269815b4a3b81cdaf7f447a518f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:24.942441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc5e269815b4a3b81cdaf7f447a518f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:24.998051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1137/2282] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:24 2022] DELETE /identity/v3/users/bcc5e269815b4a3b81cdaf7f447a518f => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:25.005392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0f958-f8a5-4ae7-bc53-c4addd6ae9b0 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:25.006300 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0f958-f8a5-4ae7-bc53-c4addd6ae9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:25.007821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0f958-f8a5-4ae7-bc53-c4addd6ae9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:25.050886 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-9fc0f958-f8a5-4ae7-bc53-c4addd6ae9b0 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:25.053090 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9fc0f958-f8a5-4ae7-bc53-c4addd6ae9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:25.138452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:25.141021 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:25.267887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1146/2283] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:25.290457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:25.299700 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0f958-f8a5-4ae7-bc53-c4addd6ae9b0 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:25.302482 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9fc0f958-f8a5-4ae7-bc53-c4addd6ae9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:25.428936 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bda2f3-d0b1-446e-b621-aba192603e55 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:25.480601 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:25.481544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:25.482193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:25.482844 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] PATH_INFO: `/v3/users/60581a263dd84eec82e30155f4156dee` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:25.489950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60581a263dd84eec82e30155f4156dee)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:25.495247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:25.513114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0f958-f8a5-4ae7-bc53-c4addd6ae9b0 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:25.538790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c75d20-ab0f-490e-9589-6d70eeeb7b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:25.540638 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c75d20-ab0f-490e-9589-6d70eeeb7b55 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:25.541176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c75d20-ab0f-490e-9589-6d70eeeb7b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:25.541658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c75d20-ab0f-490e-9589-6d70eeeb7b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:25.558111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c75d20-ab0f-490e-9589-6d70eeeb7b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:25.560654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c75d20-ab0f-490e-9589-6d70eeeb7b55 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:25.593744 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1147/2284] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:25.639233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60581a263dd84eec82e30155f4156dee'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:25.652715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60581a263dd84eec82e30155f4156dee'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:25.653489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60581a263dd84eec82e30155f4156dee'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:25.654707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60581a263dd84eec82e30155f4156dee'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:25.674369 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60581a263dd84eec82e30155f4156dee'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:25.697317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1138/2285] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:25 2022] DELETE /identity/v3/users/60581a263dd84eec82e30155f4156dee => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:25.702211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9fbcf-3bd6-4a2a-b89c-229c22d27624 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:25.702885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9fbcf-3bd6-4a2a-b89c-229c22d27624 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:25.703661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9fbcf-3bd6-4a2a-b89c-229c22d27624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:25.747693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-1fb9fbcf-3bd6-4a2a-b89c-229c22d27624 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:25.751737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1fb9fbcf-3bd6-4a2a-b89c-229c22d27624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:25.828901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:25.861275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1148/2286] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:25.875798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:25.878619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:26.057188 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac2865-eff6-4b1f-9674-6ca26a98e7a3 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:26.084087 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b4793-16e0-4cc9-9d91-4913920440e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:26.084747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b4793-16e0-4cc9-9d91-4913920440e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:26.085026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b4793-16e0-4cc9-9d91-4913920440e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:26.085284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b4793-16e0-4cc9-9d91-4913920440e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:26.097557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b4793-16e0-4cc9-9d91-4913920440e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:26.100495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b4793-16e0-4cc9-9d91-4913920440e3 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:26.128085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1139/2287] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:26.166549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9fbcf-3bd6-4a2a-b89c-229c22d27624 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:26.169118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1fb9fbcf-3bd6-4a2a-b89c-229c22d27624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:26.193093 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9fbcf-3bd6-4a2a-b89c-229c22d27624 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:26.201543 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:26.202129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:26.207756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:26.208248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] PATH_INFO: `/v3/projects/41ecb37e0e2a41ad8bb21847629b94a1` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:26.210659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ecb37e0e2a41ad8bb21847629b94a1)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:26.214317 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:26.374863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ecb37e0e2a41ad8bb21847629b94a1', 'request_id': 'req-de902c02-a1e5-423a-9589-f3a8f119c42b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.390691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ecb37e0e2a41ad8bb21847629b94a1', 'request_id': 'req-de902c02-a1e5-423a-9589-f3a8f119c42b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.409797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ecb37e0e2a41ad8bb21847629b94a1', 'request_id': 'req-de902c02-a1e5-423a-9589-f3a8f119c42b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.441161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] The token cache is being invalidate because project 41ecb37e0e2a41ad8bb21847629b94a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:26.441729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.444328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.448073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1149/2288] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:26 2022] DELETE /identity/v3/projects/41ecb37e0e2a41ad8bb21847629b94a1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:26.453482 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b4793-16e0-4cc9-9d91-4913920440e3 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:26.468917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b4793-16e0-4cc9-9d91-4913920440e3 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:26.576932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:26.593201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873260c-d70d-468d-852e-7b54002b04b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:26.594413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873260c-d70d-468d-852e-7b54002b04b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:26.594819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873260c-d70d-468d-852e-7b54002b04b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:26.595202 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873260c-d70d-468d-852e-7b54002b04b5 admin admin] PATH_INFO: `/v3/users/50527af40b2f466097bb9bdc4a652e3d` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:26.598808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873260c-d70d-468d-852e-7b54002b04b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50527af40b2f466097bb9bdc4a652e3d)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:26.602163 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873260c-d70d-468d-852e-7b54002b04b5 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:26.713942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8873260c-d70d-468d-852e-7b54002b04b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50527af40b2f466097bb9bdc4a652e3d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.735761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8873260c-d70d-468d-852e-7b54002b04b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50527af40b2f466097bb9bdc4a652e3d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.737045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8873260c-d70d-468d-852e-7b54002b04b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50527af40b2f466097bb9bdc4a652e3d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.738166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8873260c-d70d-468d-852e-7b54002b04b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50527af40b2f466097bb9bdc4a652e3d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.766308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8873260c-d70d-468d-852e-7b54002b04b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50527af40b2f466097bb9bdc4a652e3d'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.814827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:26.816554 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:26.817642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1140/2289] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:26 2022] DELETE /identity/v3/users/50527af40b2f466097bb9bdc4a652e3d => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:26.872181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de902c02-a1e5-423a-9589-f3a8f119c42b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:27.075020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:27.076188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:27.076751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:27.077286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] PATH_INFO: `/v3/projects/4de7d1725db046779e53b8ef175bb49d` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:27.080786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de7d1725db046779e53b8ef175bb49d)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:27.084174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:27.175335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de7d1725db046779e53b8ef175bb49d', 'request_id': 'req-1df3aa3d-541f-4a13-bebf-a5f396bd8720'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.187427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de7d1725db046779e53b8ef175bb49d', 'request_id': 'req-1df3aa3d-541f-4a13-bebf-a5f396bd8720'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.204344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de7d1725db046779e53b8ef175bb49d', 'request_id': 'req-1df3aa3d-541f-4a13-bebf-a5f396bd8720'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.209419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:27.227461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] The token cache is being invalidate because project 4de7d1725db046779e53b8ef175bb49d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:27.229828 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.230782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.233552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1150/2290] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:26 2022] DELETE /identity/v3/projects/4de7d1725db046779e53b8ef175bb49d => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:27.240199 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93162a-5741-4fb7-961b-db30a9dfb3d1 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:27.240510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93162a-5741-4fb7-961b-db30a9dfb3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:27.240756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93162a-5741-4fb7-961b-db30a9dfb3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:27.285299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-1c93162a-5741-4fb7-961b-db30a9dfb3d1 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:27.287130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1c93162a-5741-4fb7-961b-db30a9dfb3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:27.335934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:27.338609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:27.540990 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3aa3d-541f-4a13-bebf-a5f396bd8720 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:27.548053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:27.548870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:27.549162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:27.549420 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] PATH_INFO: `/v3/projects/0c65a92eef9744e7ba23b4bb06dd1c81` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:27.551259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c65a92eef9744e7ba23b4bb06dd1c81)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:27.553537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:27.575882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1141/2291] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:27.591533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50386a-183b-4cad-bf9c-e396fa79b73d None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:27.595628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50386a-183b-4cad-bf9c-e396fa79b73d None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:27.597222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50386a-183b-4cad-bf9c-e396fa79b73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:27.660797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c65a92eef9744e7ba23b4bb06dd1c81', 'request_id': 'req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.664888 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-2c50386a-183b-4cad-bf9c-e396fa79b73d None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:27.667912 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2c50386a-183b-4cad-bf9c-e396fa79b73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:27.697597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c65a92eef9744e7ba23b4bb06dd1c81', 'request_id': 'req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.716022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c65a92eef9744e7ba23b4bb06dd1c81', 'request_id': 'req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.732276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] The token cache is being invalidate because project 0c65a92eef9744e7ba23b4bb06dd1c81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:27.732729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.733835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.739474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1151/2292] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:27 2022] DELETE /identity/v3/projects/0c65a92eef9744e7ba23b4bb06dd1c81 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:27.741373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:27.744996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:27.879523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1142/2293] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:27.881908 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50386a-183b-4cad-bf9c-e396fa79b73d None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:27.884353 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2c50386a-183b-4cad-bf9c-e396fa79b73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:27.907726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50386a-183b-4cad-bf9c-e396fa79b73d None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:27.907977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d567e1a-bee6-4cdc-8996-fd452c51b3b5 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:27.914396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:27.914900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:27.915304 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:27.915550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:27.915637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:27.915834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:27.915914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] PATH_INFO: `/v3/users/4133f0859be546269c2b5b1a5418d8c4` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:27.916272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] PATH_INFO: `/v3/users/40b34b8d24b540f3a72da2b1eb4c37de` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:27.917168 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4133f0859be546269c2b5b1a5418d8c4)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:27.917306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=40b34b8d24b540f3a72da2b1eb4c37de)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:27.919486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:27.919648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:27.934609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:27.957021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:27.982849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4133f0859be546269c2b5b1a5418d8c4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.986239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40b34b8d24b540f3a72da2b1eb4c37de'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.991636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4133f0859be546269c2b5b1a5418d8c4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.992458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4133f0859be546269c2b5b1a5418d8c4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.992983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4133f0859be546269c2b5b1a5418d8c4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.002905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40b34b8d24b540f3a72da2b1eb4c37de'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.008923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4133f0859be546269c2b5b1a5418d8c4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.013119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40b34b8d24b540f3a72da2b1eb4c37de'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.023873 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40b34b8d24b540f3a72da2b1eb4c37de'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.025560 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40b34b8d24b540f3a72da2b1eb4c37de'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.027169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1143/2294] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:27 2022] DELETE /identity/v3/users/4133f0859be546269c2b5b1a5418d8c4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:28.028614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:28.033846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:28.049501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1152/2295] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:27 2022] DELETE /identity/v3/users/40b34b8d24b540f3a72da2b1eb4c37de => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:28.049501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:28.054860 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:28.162461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec2158-b7d2-4c5e-a2fb-fa22549aca6d admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:28.163556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd4c90-0b3b-4bf4-9584-3af2c6ceab75 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:28.172668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:28.173550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bccaf-833a-49d1-a0df-27dc019d107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:28.173762 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.174116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.174594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] PATH_INFO: `/v3/projects/4f63c452341f48c5ab62568c94e3cdfe` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.174766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bccaf-833a-49d1-a0df-27dc019d107c admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.175170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bccaf-833a-49d1-a0df-27dc019d107c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.175635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bccaf-833a-49d1-a0df-27dc019d107c admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.176848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f63c452341f48c5ab62568c94e3cdfe)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:28.177049 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bccaf-833a-49d1-a0df-27dc019d107c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:28.177428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bccaf-833a-49d1-a0df-27dc019d107c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:28.180283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:28.180847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bccaf-833a-49d1-a0df-27dc019d107c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:28.208076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1153/2296] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:28.210822 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd440d0-eb96-4fba-9ab3-f644070325d6 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.211472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd440d0-eb96-4fba-9ab3-f644070325d6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.212270 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd440d0-eb96-4fba-9ab3-f644070325d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.272162 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7dd440d0-eb96-4fba-9ab3-f644070325d6 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:28.274281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7dd440d0-eb96-4fba-9ab3-f644070325d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:28.300844 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f63c452341f48c5ab62568c94e3cdfe', 'request_id': 'req-e99c5f07-080e-4661-9109-a8a71b3f09ef'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.314273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f63c452341f48c5ab62568c94e3cdfe', 'request_id': 'req-e99c5f07-080e-4661-9109-a8a71b3f09ef'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.324843 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f63c452341f48c5ab62568c94e3cdfe', 'request_id': 'req-e99c5f07-080e-4661-9109-a8a71b3f09ef'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.349302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] The token cache is being invalidate because project 4f63c452341f48c5ab62568c94e3cdfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:28.349866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.351047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.354033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1144/2297] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:28 2022] DELETE /identity/v3/projects/4f63c452341f48c5ab62568c94e3cdfe => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:28.355479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:28.358912 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:28.406614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:28.431574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1154/2298] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:28.437951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd440d0-eb96-4fba-9ab3-f644070325d6 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:28.442400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7dd440d0-eb96-4fba-9ab3-f644070325d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:28.460461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd440d0-eb96-4fba-9ab3-f644070325d6 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:28.469685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:28.470640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.471038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.471534 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] PATH_INFO: `/v3/users/1d34f688d0b94401a1907e9845a61fc5` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.474389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d34f688d0b94401a1907e9845a61fc5)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:28.477987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:28.544193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c5f07-080e-4661-9109-a8a71b3f09ef admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:28.565986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a828c1b-412c-4c2f-b813-16e94868c41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:28.569270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a828c1b-412c-4c2f-b813-16e94868c41a admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.569773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a828c1b-412c-4c2f-b813-16e94868c41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.570672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a828c1b-412c-4c2f-b813-16e94868c41a admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.572761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a828c1b-412c-4c2f-b813-16e94868c41a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:28.580042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a828c1b-412c-4c2f-b813-16e94868c41a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:28.587692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:28.612299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d34f688d0b94401a1907e9845a61fc5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.641279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d34f688d0b94401a1907e9845a61fc5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.666163 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d34f688d0b94401a1907e9845a61fc5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.684772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d34f688d0b94401a1907e9845a61fc5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.686190 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d34f688d0b94401a1907e9845a61fc5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:28.693131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1145/2299] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:28 2022] POST /identity/v3/projects => generated 373 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:28.698622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1155/2300] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:28 2022] DELETE /identity/v3/users/1d34f688d0b94401a1907e9845a61fc5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:28.700101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:28.700618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534efef5-d795-43d9-a32e-35b0e0a10d58 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.701123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534efef5-d795-43d9-a32e-35b0e0a10d58 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.701559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534efef5-d795-43d9-a32e-35b0e0a10d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.704279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:28.741907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-534efef5-d795-43d9-a32e-35b0e0a10d58 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:28.744249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-534efef5-d795-43d9-a32e-35b0e0a10d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:28.869975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a650f1-dd08-4a4e-8940-2824acb4a438 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:28.874653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe64b22-7618-4cdf-9756-ca9a993122e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:28.875481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe64b22-7618-4cdf-9756-ca9a993122e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.875767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe64b22-7618-4cdf-9756-ca9a993122e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.876124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe64b22-7618-4cdf-9756-ca9a993122e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.878377 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-bbe64b22-7618-4cdf-9756-ca9a993122e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:28.915757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1146/2301] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:28.916585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534efef5-d795-43d9-a32e-35b0e0a10d58 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:28.918153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-534efef5-d795-43d9-a32e-35b0e0a10d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:28.946650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534efef5-d795-43d9-a32e-35b0e0a10d58 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:28.952878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791b4f8-b466-4754-bc2a-0ec812999692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:28.953690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791b4f8-b466-4754-bc2a-0ec812999692 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.954940 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791b4f8-b466-4754-bc2a-0ec812999692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.955542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791b4f8-b466-4754-bc2a-0ec812999692 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.957174 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791b4f8-b466-4754-bc2a-0ec812999692 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:28.960655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791b4f8-b466-4754-bc2a-0ec812999692 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:29.000271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1147/2302] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:28 2022] POST /identity/v3/users => generated 287 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:29.001809 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f649c8d-97cb-47bb-a961-ba001476cee9 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.002143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f649c8d-97cb-47bb-a961-ba001476cee9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.002419 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f649c8d-97cb-47bb-a961-ba001476cee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.019021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe64b22-7618-4cdf-9756-ca9a993122e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:29.025961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe64b22-7618-4cdf-9756-ca9a993122e1 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:29.027983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-1f649c8d-97cb-47bb-a961-ba001476cee9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:29.029532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1f649c8d-97cb-47bb-a961-ba001476cee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:29.039055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1148/2303] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:29.039523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f649c8d-97cb-47bb-a961-ba001476cee9 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:29.040865 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1f649c8d-97cb-47bb-a961-ba001476cee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:29.052081 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1156/2304] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:29.053742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0222de4-bf57-43c6-9ff4-ea41a774bc2e None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.054272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0222de4-bf57-43c6-9ff4-ea41a774bc2e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.055365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0222de4-bf57-43c6-9ff4-ea41a774bc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.059814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f649c8d-97cb-47bb-a961-ba001476cee9 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:29.080161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:29.080161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.080161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.080161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] PATH_INFO: `/v3/projects/ebb248869d024171a9126701f6f27aea` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.080746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb248869d024171a9126701f6f27aea)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:29.080746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:29.083102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d0222de4-bf57-43c6-9ff4-ea41a774bc2e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:29.084662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d0222de4-bf57-43c6-9ff4-ea41a774bc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:29.096698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1157/2305] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:29.097977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0222de4-bf57-43c6-9ff4-ea41a774bc2e None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:29.108457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0222de4-bf57-43c6-9ff4-ea41a774bc2e None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:29.113727 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b321d-dd67-4040-8352-5df4b45c7947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:29.114353 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b321d-dd67-4040-8352-5df4b45c7947 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.114626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b321d-dd67-4040-8352-5df4b45c7947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.114893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b321d-dd67-4040-8352-5df4b45c7947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.115613 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b321d-dd67-4040-8352-5df4b45c7947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:29.117291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b321d-dd67-4040-8352-5df4b45c7947 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:29.130062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1158/2306] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:29.131126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b321d-dd67-4040-8352-5df4b45c7947 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:29.136063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3d8b321d-dd67-4040-8352-5df4b45c7947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:29.150704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b321d-dd67-4040-8352-5df4b45c7947 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:29.168703 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb248869d024171a9126701f6f27aea', 'request_id': 'req-aff33ef5-be3f-42da-ad4d-ebf5b887111b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:29.177713 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb248869d024171a9126701f6f27aea', 'request_id': 'req-aff33ef5-be3f-42da-ad4d-ebf5b887111b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:29.186352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb248869d024171a9126701f6f27aea', 'request_id': 'req-aff33ef5-be3f-42da-ad4d-ebf5b887111b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:29.204048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] The token cache is being invalidate because project ebb248869d024171a9126701f6f27aea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:29.204786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:29.205709 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:29.207842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1149/2307] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:29 2022] DELETE /identity/v3/projects/ebb248869d024171a9126701f6f27aea => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:29.212043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:29.213507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:29.220517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:29.243816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:29.262341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56636d-71dc-42c5-98b2-e0f3f2609954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:29.264064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56636d-71dc-42c5-98b2-e0f3f2609954 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.284325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56636d-71dc-42c5-98b2-e0f3f2609954 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.285260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56636d-71dc-42c5-98b2-e0f3f2609954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.290001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7b56636d-71dc-42c5-98b2-e0f3f2609954 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:29.313948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff33ef5-be3f-42da-ad4d-ebf5b887111b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:29.320289 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf809d2d-b00d-4974-9633-8ead01a05bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:29.321136 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf809d2d-b00d-4974-9633-8ead01a05bcd admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.321420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf809d2d-b00d-4974-9633-8ead01a05bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.321760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf809d2d-b00d-4974-9633-8ead01a05bcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.322684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf809d2d-b00d-4974-9633-8ead01a05bcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:29.322994 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf809d2d-b00d-4974-9633-8ead01a05bcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:29.325255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf809d2d-b00d-4974-9633-8ead01a05bcd admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:29.344200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1150/2308] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:29.344200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf809d2d-b00d-4974-9633-8ead01a05bcd admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:29.345304 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-bf809d2d-b00d-4974-9633-8ead01a05bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:29.362547 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf809d2d-b00d-4974-9633-8ead01a05bcd admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:29.369496 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f27f71-ac01-4a69-a394-c074179c1452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:29.371241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f27f71-ac01-4a69-a394-c074179c1452 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.371681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f27f71-ac01-4a69-a394-c074179c1452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.372053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f27f71-ac01-4a69-a394-c074179c1452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.372906 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f27f71-ac01-4a69-a394-c074179c1452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:29.375570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f27f71-ac01-4a69-a394-c074179c1452 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:29.389981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56636d-71dc-42c5-98b2-e0f3f2609954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:29.391189 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56636d-71dc-42c5-98b2-e0f3f2609954 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:29.397235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1151/2309] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:29.398884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0245d1-13d5-42b8-a28d-63d9baa314a5 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.399249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0245d1-13d5-42b8-a28d-63d9baa314a5 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.399581 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0245d1-13d5-42b8-a28d-63d9baa314a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.418416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1159/2310] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:29.419767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56636d-71dc-42c5-98b2-e0f3f2609954 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:29.439233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56636d-71dc-42c5-98b2-e0f3f2609954 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:29.458366 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d44f4-c3bc-4e1d-8461-8c1cc622ef66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:29.459222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d44f4-c3bc-4e1d-8461-8c1cc622ef66 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.459662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d44f4-c3bc-4e1d-8461-8c1cc622ef66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.460131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d44f4-c3bc-4e1d-8461-8c1cc622ef66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.460913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d44f4-c3bc-4e1d-8461-8c1cc622ef66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:29.461548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-4c0245d1-13d5-42b8-a28d-63d9baa314a5 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:29.463089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d44f4-c3bc-4e1d-8461-8c1cc622ef66 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:29.465420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4c0245d1-13d5-42b8-a28d-63d9baa314a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:29.486903 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1152/2311] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:29.488374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0245d1-13d5-42b8-a28d-63d9baa314a5 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:29.513586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0245d1-13d5-42b8-a28d-63d9baa314a5 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:29.520929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0f0ff-12bb-4496-8c76-49d73f6fbc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:29.521708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0f0ff-12bb-4496-8c76-49d73f6fbc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.522013 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0f0ff-12bb-4496-8c76-49d73f6fbc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.522287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0f0ff-12bb-4496-8c76-49d73f6fbc9b admin admin] PATH_INFO: `/v3/projects/089c19009d374769a0d5bd5a82acaaa7/users/e76eba6f76f84c5cb006e079aa86157a/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.553174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1160/2312] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:29 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:29.553174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649533ef-11fb-4381-83c7-016474503ffd None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.553174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649533ef-11fb-4381-83c7-016474503ffd None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.553174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649533ef-11fb-4381-83c7-016474503ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.587755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0f0ff-12bb-4496-8c76-49d73f6fbc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=089c19009d374769a0d5bd5a82acaaa7, user_id=e76eba6f76f84c5cb006e079aa86157a, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:29.591761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0f0ff-12bb-4496-8c76-49d73f6fbc9b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:29.607981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-649533ef-11fb-4381-83c7-016474503ffd None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:29.614384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-649533ef-11fb-4381-83c7-016474503ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:29.656744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1153/2313] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:29 2022] PUT /identity/v3/projects/089c19009d374769a0d5bd5a82acaaa7/users/e76eba6f76f84c5cb006e079aa86157a/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:29.658442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0f0ff-12bb-4496-8c76-49d73f6fbc9b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:29.768452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1161/2314] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:29.774117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eafec-b15c-44e4-9b85-cb88bb6128e1 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.774638 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eafec-b15c-44e4-9b85-cb88bb6128e1 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.775733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eafec-b15c-44e4-9b85-cb88bb6128e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.815657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-c21eafec-b15c-44e4-9b85-cb88bb6128e1 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:29.818336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c21eafec-b15c-44e4-9b85-cb88bb6128e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:29.834054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1162/2315] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:29.835622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eafec-b15c-44e4-9b85-cb88bb6128e1 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:29.851680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0f0ff-12bb-4496-8c76-49d73f6fbc9b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:29.853501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eafec-b15c-44e4-9b85-cb88bb6128e1 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:29.864657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d810c-f77d-4b9d-b135-f759569ee71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:29.865057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f293d-48cd-46d6-b63a-150982e3b707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:29.865218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d810c-f77d-4b9d-b135-f759569ee71c admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.865631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d810c-f77d-4b9d-b135-f759569ee71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.866007 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d810c-f77d-4b9d-b135-f759569ee71c admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.866717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d810c-f77d-4b9d-b135-f759569ee71c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:29.867027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d810c-f77d-4b9d-b135-f759569ee71c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:29.867247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f293d-48cd-46d6-b63a-150982e3b707 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.867818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f293d-48cd-46d6-b63a-150982e3b707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.868378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f293d-48cd-46d6-b63a-150982e3b707 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.868813 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d810c-f77d-4b9d-b135-f759569ee71c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:29.869638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f293d-48cd-46d6-b63a-150982e3b707 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:29.873238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f293d-48cd-46d6-b63a-150982e3b707 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:29.891461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1163/2316] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:29.894896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d810c-f77d-4b9d-b135-f759569ee71c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:29.916327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d810c-f77d-4b9d-b135-f759569ee71c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:29.931155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1154/2317] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:29 2022] POST /identity/v3/users => generated 288 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:29.932908 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f293d-48cd-46d6-b63a-150982e3b707 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:29.935383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb79a3-6302-4420-84d0-fc6e8508b5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:29.936160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb79a3-6302-4420-84d0-fc6e8508b5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:29.937128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb79a3-6302-4420-84d0-fc6e8508b5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:29.937569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb79a3-6302-4420-84d0-fc6e8508b5c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:29.938251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb79a3-6302-4420-84d0-fc6e8508b5c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:29.939147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-801f293d-48cd-46d6-b63a-150982e3b707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:29.940361 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb79a3-6302-4420-84d0-fc6e8508b5c5 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.004647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1164/2318] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:29 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:30.004647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb79a3-6302-4420-84d0-fc6e8508b5c5 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.084985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:30.106878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:30.134008 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb79a3-6302-4420-84d0-fc6e8508b5c5 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.140217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f293d-48cd-46d6-b63a-150982e3b707 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.150793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82883e3c-0108-41a5-a6fa-032c57cf4d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.151695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82883e3c-0108-41a5-a6fa-032c57cf4d4d admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.152872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82883e3c-0108-41a5-a6fa-032c57cf4d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.153513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82883e3c-0108-41a5-a6fa-032c57cf4d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.154390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82883e3c-0108-41a5-a6fa-032c57cf4d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.161681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82883e3c-0108-41a5-a6fa-032c57cf4d4d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.202825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1165/2319] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:30.204491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82883e3c-0108-41a5-a6fa-032c57cf4d4d admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.224102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82883e3c-0108-41a5-a6fa-032c57cf4d4d admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.232422 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91638fa4-6b5e-44c0-9e39-9b31441aa57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.234219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91638fa4-6b5e-44c0-9e39-9b31441aa57d admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.234864 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91638fa4-6b5e-44c0-9e39-9b31441aa57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.235342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91638fa4-6b5e-44c0-9e39-9b31441aa57d admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.236561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91638fa4-6b5e-44c0-9e39-9b31441aa57d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.242329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91638fa4-6b5e-44c0-9e39-9b31441aa57d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.274380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84f82d-9cab-4662-90f2-6e5ad075c7d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.275452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84f82d-9cab-4662-90f2-6e5ad075c7d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.275946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84f82d-9cab-4662-90f2-6e5ad075c7d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.276535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84f82d-9cab-4662-90f2-6e5ad075c7d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.281329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1166/2320] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:30 2022] POST /identity/v3/users => generated 290 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:30.283050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91638fa4-6b5e-44c0-9e39-9b31441aa57d admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.287654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-91638fa4-6b5e-44c0-9e39-9b31441aa57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:30.298012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84f82d-9cab-4662-90f2-6e5ad075c7d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.300484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84f82d-9cab-4662-90f2-6e5ad075c7d2 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.315303 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91638fa4-6b5e-44c0-9e39-9b31441aa57d admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.316092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1155/2321] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:30.317541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84f82d-9cab-4662-90f2-6e5ad075c7d2 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.319184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ed84f82d-9cab-4662-90f2-6e5ad075c7d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:30.326724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.329577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.330422 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.331197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] PATH_INFO: `/v3/projects/585255dc2f0344dbbad821bf1110cda5` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.333526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=585255dc2f0344dbbad821bf1110cda5)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.337884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84f82d-9cab-4662-90f2-6e5ad075c7d2 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.338489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.347700 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.349190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.349708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.350435 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] PATH_INFO: `/v3/projects/7dd8aa7f0b654dd1bd05af1ea56e6aee` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.352656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dd8aa7f0b654dd1bd05af1ea56e6aee)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.355025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.459813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dd8aa7f0b654dd1bd05af1ea56e6aee', 'request_id': 'req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.470532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dd8aa7f0b654dd1bd05af1ea56e6aee', 'request_id': 'req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.480191 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '585255dc2f0344dbbad821bf1110cda5', 'request_id': 'req-956c384d-43cd-4b7d-a78e-986fafc8a1cb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.482516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dd8aa7f0b654dd1bd05af1ea56e6aee', 'request_id': 'req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.503647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '585255dc2f0344dbbad821bf1110cda5', 'request_id': 'req-956c384d-43cd-4b7d-a78e-986fafc8a1cb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.504624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] The token cache is being invalidate because project 7dd8aa7f0b654dd1bd05af1ea56e6aee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:30.504937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.506535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.509244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1156/2322] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:30 2022] DELETE /identity/v3/projects/7dd8aa7f0b654dd1bd05af1ea56e6aee => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:30.510756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.516464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '585255dc2f0344dbbad821bf1110cda5', 'request_id': 'req-956c384d-43cd-4b7d-a78e-986fafc8a1cb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.517187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:30.534423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] The token cache is being invalidate because project 585255dc2f0344dbbad821bf1110cda5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:30.534741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.535504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.541116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1167/2323] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:30 2022] DELETE /identity/v3/projects/585255dc2f0344dbbad821bf1110cda5 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:30.541116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.541116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:30.651344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c384d-43cd-4b7d-a78e-986fafc8a1cb admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.658142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce8069-0b74-4ef6-b602-c8493183bc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.658460 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce8069-0b74-4ef6-b602-c8493183bc1b admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.658565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce8069-0b74-4ef6-b602-c8493183bc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.658772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce8069-0b74-4ef6-b602-c8493183bc1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.659714 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce8069-0b74-4ef6-b602-c8493183bc1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.661550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce8069-0b74-4ef6-b602-c8493183bc1b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.666714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6a6da-23b4-4c0b-8d18-49cc6fac4ef4 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.680546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1168/2324] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:30.680546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce8069-0b74-4ef6-b602-c8493183bc1b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.684181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3cca5-64e2-4d6c-9daf-92369393d1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.684974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3cca5-64e2-4d6c-9daf-92369393d1ef admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.685316 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-50ce8069-0b74-4ef6-b602-c8493183bc1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:30.685497 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3cca5-64e2-4d6c-9daf-92369393d1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.685816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3cca5-64e2-4d6c-9daf-92369393d1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.686507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3cca5-64e2-4d6c-9daf-92369393d1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.699623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3cca5-64e2-4d6c-9daf-92369393d1ef admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.723706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1157/2325] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:30.725534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48966629-e358-4037-b41a-32b7916be396 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.726035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48966629-e358-4037-b41a-32b7916be396 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.726455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48966629-e358-4037-b41a-32b7916be396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.728097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce8069-0b74-4ef6-b602-c8493183bc1b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.737350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ce605-9881-49fe-8824-d45a1ce9ef75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.738871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ce605-9881-49fe-8824-d45a1ce9ef75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.740046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ce605-9881-49fe-8824-d45a1ce9ef75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.740943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ce605-9881-49fe-8824-d45a1ce9ef75 admin admin] PATH_INFO: `/v3/users/4f7f03e1d94d405da53513b0061335fb` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.743459 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ce605-9881-49fe-8824-d45a1ce9ef75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f7f03e1d94d405da53513b0061335fb)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.751171 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ce605-9881-49fe-8824-d45a1ce9ef75 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.769791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-48966629-e358-4037-b41a-32b7916be396 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:30.772209 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-48966629-e358-4037-b41a-32b7916be396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:30.786967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1158/2326] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:30.788322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48966629-e358-4037-b41a-32b7916be396 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.804924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48966629-e358-4037-b41a-32b7916be396 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.816426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:30.817201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8f7b0-8f23-49d6-bad3-c7d91c9bd87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.818067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8f7b0-8f23-49d6-bad3-c7d91c9bd87d admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.818386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8f7b0-8f23-49d6-bad3-c7d91c9bd87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.818798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8f7b0-8f23-49d6-bad3-c7d91c9bd87d admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.819835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8f7b0-8f23-49d6-bad3-c7d91c9bd87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.821737 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8f7b0-8f23-49d6-bad3-c7d91c9bd87d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.838981 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1159/2327] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:30.848370 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8f7b0-8f23-49d6-bad3-c7d91c9bd87d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.853561 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7be8f7b0-8f23-49d6-bad3-c7d91c9bd87d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:30.869609 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-af7ce605-9881-49fe-8824-d45a1ce9ef75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f7f03e1d94d405da53513b0061335fb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.894534 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-af7ce605-9881-49fe-8824-d45a1ce9ef75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f7f03e1d94d405da53513b0061335fb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.895875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8f7b0-8f23-49d6-bad3-c7d91c9bd87d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.909208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-af7ce605-9881-49fe-8824-d45a1ce9ef75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f7f03e1d94d405da53513b0061335fb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.914105 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4897fdc-dfb7-4dda-8379-ab47231f2bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.919142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4897fdc-dfb7-4dda-8379-ab47231f2bac admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.919891 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4897fdc-dfb7-4dda-8379-ab47231f2bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.921746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4897fdc-dfb7-4dda-8379-ab47231f2bac admin admin] PATH_INFO: `/v3/projects/43adb894c667464a944b9e6da549c496/users/e7df0b1f38a549519e64fbd07cd4f319/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.924546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-af7ce605-9881-49fe-8824-d45a1ce9ef75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f7f03e1d94d405da53513b0061335fb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.925518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-af7ce605-9881-49fe-8824-d45a1ce9ef75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f7f03e1d94d405da53513b0061335fb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:30.929827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:30.940590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1169/2328] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:30 2022] DELETE /identity/v3/users/4f7f03e1d94d405da53513b0061335fb => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:30.942404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f1fb2-d3fb-4d32-9e8b-85039051198c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.942845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f1fb2-d3fb-4d32-9e8b-85039051198c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.943901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f1fb2-d3fb-4d32-9e8b-85039051198c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.971629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4897fdc-dfb7-4dda-8379-ab47231f2bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=43adb894c667464a944b9e6da549c496, user_id=e7df0b1f38a549519e64fbd07cd4f319, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.974640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ec5f1fb2-d3fb-4d32-9e8b-85039051198c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:30.976672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ec5f1fb2-d3fb-4d32-9e8b-85039051198c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:30.978053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4897fdc-dfb7-4dda-8379-ab47231f2bac admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:31.010612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1160/2329] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:30 2022] PUT /identity/v3/projects/43adb894c667464a944b9e6da549c496/users/e7df0b1f38a549519e64fbd07cd4f319/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:31.016039 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4897fdc-dfb7-4dda-8379-ab47231f2bac admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:31.027066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c4897fdc-dfb7-4dda-8379-ab47231f2bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:31.127280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1170/2330] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:31.130233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b1c1b-c1ce-40af-b707-b0b48e0414f3 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:31.130701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b1c1b-c1ce-40af-b707-b0b48e0414f3 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:31.132334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b1c1b-c1ce-40af-b707-b0b48e0414f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:31.175930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4897fdc-dfb7-4dda-8379-ab47231f2bac admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:31.183330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-3f3b1c1b-c1ce-40af-b707-b0b48e0414f3 None None] MFA Rules not processed for user `e7df0b1f38a549519e64fbd07cd4f319`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:31.185462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3f3b1c1b-c1ce-40af-b707-b0b48e0414f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:31.186476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b1db8-0ad3-4649-830d-178e7e05da5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:31.187654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b1db8-0ad3-4649-830d-178e7e05da5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:31.188172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b1db8-0ad3-4649-830d-178e7e05da5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:31.188659 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b1db8-0ad3-4649-830d-178e7e05da5d admin admin] PATH_INFO: `/v3/projects/6e0ae6dd632e46cc968bdd864a4cb717/users/0ffcf88880dd4c04ae1e80cda44e4908/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:31.218316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b1db8-0ad3-4649-830d-178e7e05da5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e0ae6dd632e46cc968bdd864a4cb717, user_id=0ffcf88880dd4c04ae1e80cda44e4908, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:31.224306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b1db8-0ad3-4649-830d-178e7e05da5d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:31.245246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1161/2331] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:31 2022] PUT /identity/v3/projects/6e0ae6dd632e46cc968bdd864a4cb717/users/0ffcf88880dd4c04ae1e80cda44e4908/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:31.246377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b1db8-0ad3-4649-830d-178e7e05da5d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:31.360701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1171/2332] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:31.361894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d56ba3-d3df-47ff-b05d-ac82b812f2e5 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:31.362178 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d56ba3-d3df-47ff-b05d-ac82b812f2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:31.362451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d56ba3-d3df-47ff-b05d-ac82b812f2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:31.400117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b0d56ba3-d3df-47ff-b05d-ac82b812f2e5 None None] MFA Rules not processed for user `0ffcf88880dd4c04ae1e80cda44e4908`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:31.400627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b0d56ba3-d3df-47ff-b05d-ac82b812f2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:31.419245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b1db8-0ad3-4649-830d-178e7e05da5d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:31.425810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:31.426764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0a73e-ce9b-4387-b2fc-eeb6cd89e9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:31.427701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0a73e-ce9b-4387-b2fc-eeb6cd89e9cb admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:31.427985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0a73e-ce9b-4387-b2fc-eeb6cd89e9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:31.428291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0a73e-ce9b-4387-b2fc-eeb6cd89e9cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:31.428973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0a73e-ce9b-4387-b2fc-eeb6cd89e9cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:31.429308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0a73e-ce9b-4387-b2fc-eeb6cd89e9cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:31.431404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0a73e-ce9b-4387-b2fc-eeb6cd89e9cb admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:31.451619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1162/2333] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:31.451619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0a73e-ce9b-4387-b2fc-eeb6cd89e9cb admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:31.453706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c3b0a73e-ce9b-4387-b2fc-eeb6cd89e9cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:31.501754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1172/2334] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:31.502752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d56ba3-d3df-47ff-b05d-ac82b812f2e5 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:31.518811 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d56ba3-d3df-47ff-b05d-ac82b812f2e5 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:31.525975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd873a2-ec38-4b96-87f1-f741ca0032b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:31.526792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd873a2-ec38-4b96-87f1-f741ca0032b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:31.527131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd873a2-ec38-4b96-87f1-f741ca0032b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:31.527455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd873a2-ec38-4b96-87f1-f741ca0032b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:31.528227 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd873a2-ec38-4b96-87f1-f741ca0032b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:31.531486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd873a2-ec38-4b96-87f1-f741ca0032b1 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:31.543641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0a73e-ce9b-4387-b2fc-eeb6cd89e9cb admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:31.551178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8d504-ef80-48fe-ab80-b8bc38011a59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:31.552030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8d504-ef80-48fe-ab80-b8bc38011a59 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:31.552202 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8d504-ef80-48fe-ab80-b8bc38011a59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:31.552373 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8d504-ef80-48fe-ab80-b8bc38011a59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:31.569768 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:31.604539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1173/2335] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:31 2022] POST /identity/v3/projects => generated 397 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:31.606652 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd873a2-ec38-4b96-87f1-f741ca0032b1 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:31.642594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8d504-ef80-48fe-ab80-b8bc38011a59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:31.644106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8d504-ef80-48fe-ab80-b8bc38011a59 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:31.654712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1163/2336] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:31.658627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8d504-ef80-48fe-ab80-b8bc38011a59 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:31.679653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8d504-ef80-48fe-ab80-b8bc38011a59 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:31.709706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd873a2-ec38-4b96-87f1-f741ca0032b1 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:31.716014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb07ec-7c09-4347-a482-409f3119423e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:31.716837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb07ec-7c09-4347-a482-409f3119423e admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:31.718393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb07ec-7c09-4347-a482-409f3119423e admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:31.718777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb07ec-7c09-4347-a482-409f3119423e admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:31.720139 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb07ec-7c09-4347-a482-409f3119423e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:31.721774 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb07ec-7c09-4347-a482-409f3119423e admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:31.763690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1174/2337] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:31 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:31.766642 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb07ec-7c09-4347-a482-409f3119423e admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:31.847254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a611de-4c4e-4c87-8592-0208161d65ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:31.847892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a611de-4c4e-4c87-8592-0208161d65ac admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:31.848193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a611de-4c4e-4c87-8592-0208161d65ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:31.848463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a611de-4c4e-4c87-8592-0208161d65ac admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:31.850882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a611de-4c4e-4c87-8592-0208161d65ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:31.855175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a611de-4c4e-4c87-8592-0208161d65ac admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:31.887254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb07ec-7c09-4347-a482-409f3119423e admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:31.894411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898494b-e017-4841-8d2a-ece7fc62549c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:31.895182 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898494b-e017-4841-8d2a-ece7fc62549c admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:31.895826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898494b-e017-4841-8d2a-ece7fc62549c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:31.896362 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898494b-e017-4841-8d2a-ece7fc62549c admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:31.897076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898494b-e017-4841-8d2a-ece7fc62549c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:31.899559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898494b-e017-4841-8d2a-ece7fc62549c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:31.908740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1164/2338] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:31 2022] POST /identity/v3/users => generated 299 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:31.913584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a611de-4c4e-4c87-8592-0208161d65ac admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:31.930846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a611de-4c4e-4c87-8592-0208161d65ac admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:31.938756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8a443-a1d5-4a88-925e-932401587895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:31.939729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a443-a1d5-4a88-925e-932401587895 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:31.940242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1175/2339] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:31 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:31.940360 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a443-a1d5-4a88-925e-932401587895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:31.940835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a443-a1d5-4a88-925e-932401587895 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:31.941990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8a443-a1d5-4a88-925e-932401587895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:31.944610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8a443-a1d5-4a88-925e-932401587895 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:31.955253 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898494b-e017-4841-8d2a-ece7fc62549c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:31.964458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1165/2340] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:31.971583 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8a443-a1d5-4a88-925e-932401587895 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:31.987373 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8a443-a1d5-4a88-925e-932401587895 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:31.989934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898494b-e017-4841-8d2a-ece7fc62549c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:31.995407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc17e2-a1bc-40a0-af04-b070c0617804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:31.996128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc17e2-a1bc-40a0-af04-b070c0617804 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:31.996386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc17e2-a1bc-40a0-af04-b070c0617804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:31.996688 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc17e2-a1bc-40a0-af04-b070c0617804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:31.998403 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc17e2-a1bc-40a0-af04-b070c0617804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:32.000109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc17e2-a1bc-40a0-af04-b070c0617804 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:32.006126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ab7a2-20f7-466e-b99a-9fcb06e1afe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:32.007168 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ab7a2-20f7-466e-b99a-9fcb06e1afe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:32.007677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ab7a2-20f7-466e-b99a-9fcb06e1afe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:32.008335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ab7a2-20f7-466e-b99a-9fcb06e1afe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:32.010677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ab7a2-20f7-466e-b99a-9fcb06e1afe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:32.016414 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ab7a2-20f7-466e-b99a-9fcb06e1afe6 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:32.018168 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1166/2341] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:32.023913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc17e2-a1bc-40a0-af04-b070c0617804 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:32.040088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1176/2342] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:32.044906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ab7a2-20f7-466e-b99a-9fcb06e1afe6 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:32.050366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc17e2-a1bc-40a0-af04-b070c0617804 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:32.057922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:32.061534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c20f6-2ead-4b00-a8c3-246315f1f0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:32.063904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c20f6-2ead-4b00-a8c3-246315f1f0bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:32.064405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c20f6-2ead-4b00-a8c3-246315f1f0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:32.064852 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c20f6-2ead-4b00-a8c3-246315f1f0bd admin admin] PATH_INFO: `/v3/projects/681cbb7fcb00427c8fca3cfee8718e23/users/fc3a7e23a4c04d9fb48d4b3b95091006/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:32.070538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ab7a2-20f7-466e-b99a-9fcb06e1afe6 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:32.089494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537626f4-a5d8-431c-9572-78ff431a3e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:32.090743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537626f4-a5d8-431c-9572-78ff431a3e70 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:32.091050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537626f4-a5d8-431c-9572-78ff431a3e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:32.091295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537626f4-a5d8-431c-9572-78ff431a3e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:32.091967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537626f4-a5d8-431c-9572-78ff431a3e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:32.096622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537626f4-a5d8-431c-9572-78ff431a3e70 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:32.101862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c20f6-2ead-4b00-a8c3-246315f1f0bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=681cbb7fcb00427c8fca3cfee8718e23, user_id=fc3a7e23a4c04d9fb48d4b3b95091006, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:32.105393 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c20f6-2ead-4b00-a8c3-246315f1f0bd admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:32.124299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1177/2343] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:32.126167 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537626f4-a5d8-431c-9572-78ff431a3e70 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:32.150881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537626f4-a5d8-431c-9572-78ff431a3e70 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:32.167941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1167/2344] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:32 2022] PUT /identity/v3/projects/681cbb7fcb00427c8fca3cfee8718e23/users/fc3a7e23a4c04d9fb48d4b3b95091006/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:32.183359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e3296-ae27-4de1-96cc-30ad86354d3f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:32.183685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e3296-ae27-4de1-96cc-30ad86354d3f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:32.184155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e3296-ae27-4de1-96cc-30ad86354d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:32.296095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-0f5e3296-ae27-4de1-96cc-30ad86354d3f None None] MFA Rules not processed for user `fc3a7e23a4c04d9fb48d4b3b95091006`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:32.299511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0f5e3296-ae27-4de1-96cc-30ad86354d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:32.340863 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec1cc3-8bc2-4a32-8459-17aaba8ff94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:32.341695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec1cc3-8bc2-4a32-8459-17aaba8ff94a admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:32.342065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec1cc3-8bc2-4a32-8459-17aaba8ff94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:32.342768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec1cc3-8bc2-4a32-8459-17aaba8ff94a admin admin] PATH_INFO: `/v3/projects/6da0f07d4605434daf5eb09a872f84b8/users/597590bafe384dad83192849b9dfd209/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:32.382540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec1cc3-8bc2-4a32-8459-17aaba8ff94a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da0f07d4605434daf5eb09a872f84b8, user_id=597590bafe384dad83192849b9dfd209, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:32.385981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec1cc3-8bc2-4a32-8459-17aaba8ff94a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:32.427552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1178/2345] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:32 2022] PUT /identity/v3/projects/6da0f07d4605434daf5eb09a872f84b8/users/597590bafe384dad83192849b9dfd209/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:32.430992 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec1cc3-8bc2-4a32-8459-17aaba8ff94a admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:32.438120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:32.442981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-eeec1cc3-8bc2-4a32-8459-17aaba8ff94a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:32.509153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1168/2346] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:32.509153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770711f-8a1e-48ed-a57f-407cb113d6bc None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:32.509153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770711f-8a1e-48ed-a57f-407cb113d6bc None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:32.509153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770711f-8a1e-48ed-a57f-407cb113d6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:32.591137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d770711f-8a1e-48ed-a57f-407cb113d6bc None None] MFA Rules not processed for user `597590bafe384dad83192849b9dfd209`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:32.594017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d770711f-8a1e-48ed-a57f-407cb113d6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:32.704078 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec1cc3-8bc2-4a32-8459-17aaba8ff94a admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:32.750109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:32.758609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1169/2347] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:32.759144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:32.759399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:32.759603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] PATH_INFO: `/v3/projects/d2813420736a4b81915e556a1c2a266b` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:32.760969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770711f-8a1e-48ed-a57f-407cb113d6bc None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:32.774926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2813420736a4b81915e556a1c2a266b)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:32.784743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:32.856056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770711f-8a1e-48ed-a57f-407cb113d6bc None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:32.864213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0e8a6-e8aa-481b-82ed-8f0f16058ef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:32.865577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0e8a6-e8aa-481b-82ed-8f0f16058ef2 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:32.865915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0e8a6-e8aa-481b-82ed-8f0f16058ef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:32.866240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0e8a6-e8aa-481b-82ed-8f0f16058ef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:32.928701 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2813420736a4b81915e556a1c2a266b', 'request_id': 'req-41d07582-d704-4088-b37f-edb2c2c4399b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:32.940439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2813420736a4b81915e556a1c2a266b', 'request_id': 'req-41d07582-d704-4088-b37f-edb2c2c4399b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:32.958748 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2813420736a4b81915e556a1c2a266b', 'request_id': 'req-41d07582-d704-4088-b37f-edb2c2c4399b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:32.988622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] The token cache is being invalidate because project d2813420736a4b81915e556a1c2a266b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:32.989948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:32.991227 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0e8a6-e8aa-481b-82ed-8f0f16058ef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:32.992820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0e8a6-e8aa-481b-82ed-8f0f16058ef2 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:32.993282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:33.002737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1179/2348] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:32 2022] DELETE /identity/v3/projects/d2813420736a4b81915e556a1c2a266b => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:33.002737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:33.006417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:33.013803 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1170/2349] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:33.015964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9fdd6-7f66-45de-afe8-5ad0d5ee46e3 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:33.016346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9fdd6-7f66-45de-afe8-5ad0d5ee46e3 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:33.016718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9fdd6-7f66-45de-afe8-5ad0d5ee46e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:33.035148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:33.049291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-efd9fdd6-7f66-45de-afe8-5ad0d5ee46e3 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:33.051103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-efd9fdd6-7f66-45de-afe8-5ad0d5ee46e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:33.096590 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:33.163676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d07582-d704-4088-b37f-edb2c2c4399b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:33.170038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c781502-c55b-4c6e-88c4-2a59d2bb73c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:33.171326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c781502-c55b-4c6e-88c4-2a59d2bb73c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:33.171665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c781502-c55b-4c6e-88c4-2a59d2bb73c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:33.172608 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c781502-c55b-4c6e-88c4-2a59d2bb73c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:33.173849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c781502-c55b-4c6e-88c4-2a59d2bb73c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:33.188733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c781502-c55b-4c6e-88c4-2a59d2bb73c6 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:33.191137 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1171/2350] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:33.199882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b85450-3506-4155-a764-7ec3c0e29368 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:33.200347 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b85450-3506-4155-a764-7ec3c0e29368 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:33.200635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b85450-3506-4155-a764-7ec3c0e29368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:33.272045 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1180/2351] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:32 2022] POST /identity/v3/projects => generated 377 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:33.275174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c781502-c55b-4c6e-88c4-2a59d2bb73c6 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:33.287078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-63b85450-3506-4155-a764-7ec3c0e29368 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:33.288425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-63b85450-3506-4155-a764-7ec3c0e29368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:33.400474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1172/2352] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:33.401464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b85450-3506-4155-a764-7ec3c0e29368 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:33.423237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b85450-3506-4155-a764-7ec3c0e29368 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:33.430606 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d820bd-1117-44e1-ad8f-ae9d01069550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:33.430992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d820bd-1117-44e1-ad8f-ae9d01069550 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:33.431587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d820bd-1117-44e1-ad8f-ae9d01069550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:33.431894 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d820bd-1117-44e1-ad8f-ae9d01069550 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:33.432597 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d820bd-1117-44e1-ad8f-ae9d01069550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:33.433082 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d820bd-1117-44e1-ad8f-ae9d01069550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:33.435203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d820bd-1117-44e1-ad8f-ae9d01069550 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:33.458516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1173/2353] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:33.459832 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d820bd-1117-44e1-ad8f-ae9d01069550 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:33.471821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d820bd-1117-44e1-ad8f-ae9d01069550 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:33.477154 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2bc42-64f8-4ee5-8fb8-15365ec38a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:33.478770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2bc42-64f8-4ee5-8fb8-15365ec38a54 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:33.478770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2bc42-64f8-4ee5-8fb8-15365ec38a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:33.478770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2bc42-64f8-4ee5-8fb8-15365ec38a54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:33.479260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2bc42-64f8-4ee5-8fb8-15365ec38a54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:33.481791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2bc42-64f8-4ee5-8fb8-15365ec38a54 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:33.505687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c781502-c55b-4c6e-88c4-2a59d2bb73c6 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:33.512488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb3adc-eed6-4f55-9a87-5f6269fd4879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:33.513140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb3adc-eed6-4f55-9a87-5f6269fd4879 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:33.513417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb3adc-eed6-4f55-9a87-5f6269fd4879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:33.513693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb3adc-eed6-4f55-9a87-5f6269fd4879 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:33.514381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb3adc-eed6-4f55-9a87-5f6269fd4879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:33.516818 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb3adc-eed6-4f55-9a87-5f6269fd4879 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:33.536980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1174/2354] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:33 2022] POST /identity/v3/projects => generated 363 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:33.539559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2bc42-64f8-4ee5-8fb8-15365ec38a54 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:33.550050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1181/2355] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:33 2022] POST /identity/v3/users => generated 295 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:33.556298 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb3adc-eed6-4f55-9a87-5f6269fd4879 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:33.589735 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb3adc-eed6-4f55-9a87-5f6269fd4879 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:33.719889 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2bc42-64f8-4ee5-8fb8-15365ec38a54 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:33.726524 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d97e45-ffb5-41ca-a254-99bd839b5741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:33.727383 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d97e45-ffb5-41ca-a254-99bd839b5741 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:33.728255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d97e45-ffb5-41ca-a254-99bd839b5741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:33.728874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d97e45-ffb5-41ca-a254-99bd839b5741 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:33.729849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d97e45-ffb5-41ca-a254-99bd839b5741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:33.733571 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d97e45-ffb5-41ca-a254-99bd839b5741 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:33.747790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37975f0-313f-472b-8ad6-e80db9fe0585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:33.752897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37975f0-313f-472b-8ad6-e80db9fe0585 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:33.753236 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37975f0-313f-472b-8ad6-e80db9fe0585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:33.753556 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37975f0-313f-472b-8ad6-e80db9fe0585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:33.754311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37975f0-313f-472b-8ad6-e80db9fe0585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:33.758403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37975f0-313f-472b-8ad6-e80db9fe0585 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:33.772681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1175/2356] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:33 2022] POST /identity/v3/users => generated 282 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:33.773862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d97e45-ffb5-41ca-a254-99bd839b5741 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:33.787776 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d97e45-ffb5-41ca-a254-99bd839b5741 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:33.792534 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1182/2357] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:33.805354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37975f0-313f-472b-8ad6-e80db9fe0585 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:33.818618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856e19c-ba4c-404a-bf31-f57ca971f02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:33.819433 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856e19c-ba4c-404a-bf31-f57ca971f02b admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:33.819782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856e19c-ba4c-404a-bf31-f57ca971f02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:33.820486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856e19c-ba4c-404a-bf31-f57ca971f02b admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:33.822579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856e19c-ba4c-404a-bf31-f57ca971f02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:33.824935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856e19c-ba4c-404a-bf31-f57ca971f02b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:33.833686 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37975f0-313f-472b-8ad6-e80db9fe0585 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:33.843165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1176/2358] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:33.849474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856e19c-ba4c-404a-bf31-f57ca971f02b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:33.853601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbd254-1fa0-4d51-b163-20ff941ebd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:33.854900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd254-1fa0-4d51-b163-20ff941ebd73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:33.855477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd254-1fa0-4d51-b163-20ff941ebd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:33.856388 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd254-1fa0-4d51-b163-20ff941ebd73 admin admin] PATH_INFO: `/v3/projects/62571f2ead874cc6bcb08054e3888aba/users/431f7d0f412b4d768f720b7b532b98c4/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:33.880632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856e19c-ba4c-404a-bf31-f57ca971f02b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:33.899311 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c5eb9-84be-4ccd-a5c0-d2b739bcef9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:33.900047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c5eb9-84be-4ccd-a5c0-d2b739bcef9b admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:33.900333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c5eb9-84be-4ccd-a5c0-d2b739bcef9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:33.900626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c5eb9-84be-4ccd-a5c0-d2b739bcef9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:33.901548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:33.903217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c5eb9-84be-4ccd-a5c0-d2b739bcef9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:33.906936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbd254-1fa0-4d51-b163-20ff941ebd73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62571f2ead874cc6bcb08054e3888aba, user_id=431f7d0f412b4d768f720b7b532b98c4, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:33.912312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c5eb9-84be-4ccd-a5c0-d2b739bcef9b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:33.914929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbd254-1fa0-4d51-b163-20ff941ebd73 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:33.944391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1177/2359] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:33.950165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c5eb9-84be-4ccd-a5c0-d2b739bcef9b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:33.965749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:33.971904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1183/2360] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:33 2022] PUT /identity/v3/projects/62571f2ead874cc6bcb08054e3888aba/users/431f7d0f412b4d768f720b7b532b98c4/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:33.980526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbd254-1fa0-4d51-b163-20ff941ebd73 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:33.980741 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c5eb9-84be-4ccd-a5c0-d2b739bcef9b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:34.177442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bc40e-71d2-4fea-86a7-78d12f61d847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:34.178712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bc40e-71d2-4fea-86a7-78d12f61d847 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:34.179232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bc40e-71d2-4fea-86a7-78d12f61d847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:34.179807 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bc40e-71d2-4fea-86a7-78d12f61d847 admin admin] PATH_INFO: `/v3/projects/a4e3c9191c3542bbb022d1ffdf052a92/users/c654d71744924651a9928fb428d0c93c/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:34.203973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbd254-1fa0-4d51-b163-20ff941ebd73 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:34.252688 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662bc40e-71d2-4fea-86a7-78d12f61d847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e3c9191c3542bbb022d1ffdf052a92, user_id=c654d71744924651a9928fb428d0c93c, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:34.255397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662bc40e-71d2-4fea-86a7-78d12f61d847 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:34.257884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc952a-8008-4436-a84c-62dcef1f1424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:34.261995 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc952a-8008-4436-a84c-62dcef1f1424 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:34.273629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc952a-8008-4436-a84c-62dcef1f1424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:34.275027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc952a-8008-4436-a84c-62dcef1f1424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:34.276194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc952a-8008-4436-a84c-62dcef1f1424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:34.301613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1178/2361] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:33 2022] PUT /identity/v3/projects/a4e3c9191c3542bbb022d1ffdf052a92/users/c654d71744924651a9928fb428d0c93c/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:34.303128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178da09-f5e3-4e25-a292-4ff469570d14 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:34.303412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178da09-f5e3-4e25-a292-4ff469570d14 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:34.303751 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178da09-f5e3-4e25-a292-4ff469570d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:34.312489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc952a-8008-4436-a84c-62dcef1f1424 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:34.466459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-f178da09-f5e3-4e25-a292-4ff469570d14 None None] MFA Rules not processed for user `c654d71744924651a9928fb428d0c93c`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:34.469332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1184/2362] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 487 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:34.470860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f178da09-f5e3-4e25-a292-4ff469570d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:34.473506 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc952a-8008-4436-a84c-62dcef1f1424 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:34.497551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc952a-8008-4436-a84c-62dcef1f1424 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:34.731781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1179/2363] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:34.747297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178da09-f5e3-4e25-a292-4ff469570d14 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:34.755496 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f178da09-f5e3-4e25-a292-4ff469570d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:34.770472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73dce4-2d19-492e-b802-53ab03ad68ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:34.770472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73dce4-2d19-492e-b802-53ab03ad68ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:34.770808 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73dce4-2d19-492e-b802-53ab03ad68ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:34.770973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73dce4-2d19-492e-b802-53ab03ad68ff admin admin] PATH_INFO: `/v3/projects/62571f2ead874cc6bcb08054e3888aba/users/431f7d0f412b4d768f720b7b532b98c4/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:34.773557 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73dce4-2d19-492e-b802-53ab03ad68ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=62571f2ead874cc6bcb08054e3888aba, user_id=431f7d0f412b4d768f720b7b532b98c4, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:34.776818 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73dce4-2d19-492e-b802-53ab03ad68ff admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:34.800885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1185/2364] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:34 2022] PUT /identity/v3/projects/62571f2ead874cc6bcb08054e3888aba/users/431f7d0f412b4d768f720b7b532b98c4/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:34.813888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdb495-73d0-4c89-b954-c08fa9167bc1 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:34.814404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdb495-73d0-4c89-b954-c08fa9167bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:34.815975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdb495-73d0-4c89-b954-c08fa9167bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:34.899057 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-5ffdb495-73d0-4c89-b954-c08fa9167bc1 None None] MFA Rules not processed for user `431f7d0f412b4d768f720b7b532b98c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:34.900314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5ffdb495-73d0-4c89-b954-c08fa9167bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:34.922095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178da09-f5e3-4e25-a292-4ff469570d14 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:34.988645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:35.022983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839295e-e493-4edd-8494-5412ad767fe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:35.023617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839295e-e493-4edd-8494-5412ad767fe7 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:35.024034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839295e-e493-4edd-8494-5412ad767fe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:35.024367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839295e-e493-4edd-8494-5412ad767fe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:35.036457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1186/2365] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:35.098792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdb495-73d0-4c89-b954-c08fa9167bc1 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:35.101118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5ffdb495-73d0-4c89-b954-c08fa9167bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:35.127223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839295e-e493-4edd-8494-5412ad767fe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:35.127917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdb495-73d0-4c89-b954-c08fa9167bc1 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:35.129138 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839295e-e493-4edd-8494-5412ad767fe7 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:35.134152 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d60f2-2992-49b5-92b8-2e5829be737c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:35.135329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d60f2-2992-49b5-92b8-2e5829be737c service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:35.135802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d60f2-2992-49b5-92b8-2e5829be737c service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:35.136454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d60f2-2992-49b5-92b8-2e5829be737c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:35.169811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1180/2366] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:35.169981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:35.181386 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d60f2-2992-49b5-92b8-2e5829be737c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:35.184941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d60f2-2992-49b5-92b8-2e5829be737c service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:35.231782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1187/2367] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:35.455356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839295e-e493-4edd-8494-5412ad767fe7 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:35.465663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839295e-e493-4edd-8494-5412ad767fe7 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:35.470927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a50b4e-e4e3-4efe-b74c-c3188d5deb27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:35.471559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a50b4e-e4e3-4efe-b74c-c3188d5deb27 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:35.471840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a50b4e-e4e3-4efe-b74c-c3188d5deb27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:35.472154 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a50b4e-e4e3-4efe-b74c-c3188d5deb27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:35.474252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b1a50b4e-e4e3-4efe-b74c-c3188d5deb27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:35.600698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a50b4e-e4e3-4efe-b74c-c3188d5deb27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:35.603748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a50b4e-e4e3-4efe-b74c-c3188d5deb27 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:35.616766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1181/2368] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:39.550864 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d85f1-16c5-4fac-87a3-e7ae90530d11 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.551411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d85f1-16c5-4fac-87a3-e7ae90530d11 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.551411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d85f1-16c5-4fac-87a3-e7ae90530d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.596297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-5d0d85f1-16c5-4fac-87a3-e7ae90530d11 None None] MFA Rules not processed for user `e76eba6f76f84c5cb006e079aa86157a`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:39.597477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5d0d85f1-16c5-4fac-87a3-e7ae90530d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:39.683280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1188/2369] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:39.689877 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a50b4e-e4e3-4efe-b74c-c3188d5deb27 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:39.692089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b1a50b4e-e4e3-4efe-b74c-c3188d5deb27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:39.806369 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a50b4e-e4e3-4efe-b74c-c3188d5deb27 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:39.814251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7edb0-5421-4a00-8a73-d0b2f9650574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:39.814897 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7edb0-5421-4a00-8a73-d0b2f9650574 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.815176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7edb0-5421-4a00-8a73-d0b2f9650574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.815446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7edb0-5421-4a00-8a73-d0b2f9650574 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.816141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7edb0-5421-4a00-8a73-d0b2f9650574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:39.818143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7edb0-5421-4a00-8a73-d0b2f9650574 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:39.864838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1182/2370] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:39 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:39.869376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d85f1-16c5-4fac-87a3-e7ae90530d11 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:39.884917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d85f1-16c5-4fac-87a3-e7ae90530d11 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:39.957297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbb63f-8326-41f2-bda7-b43ac8cc150c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:39.957978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbb63f-8326-41f2-bda7-b43ac8cc150c admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.958390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbb63f-8326-41f2-bda7-b43ac8cc150c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.958662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbb63f-8326-41f2-bda7-b43ac8cc150c admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.959436 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbb63f-8326-41f2-bda7-b43ac8cc150c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:39.961577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbb63f-8326-41f2-bda7-b43ac8cc150c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:39.987584 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1189/2371] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:39 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:39.992926 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7edb0-5421-4a00-8a73-d0b2f9650574 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:40.012089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7edb0-5421-4a00-8a73-d0b2f9650574 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:40.021750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de9336-3ce1-48d1-b8a9-48b5de368b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:40.022743 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de9336-3ce1-48d1-b8a9-48b5de368b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:40.023200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de9336-3ce1-48d1-b8a9-48b5de368b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:40.023643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de9336-3ce1-48d1-b8a9-48b5de368b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:40.025747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de9336-3ce1-48d1-b8a9-48b5de368b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:40.028878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de9336-3ce1-48d1-b8a9-48b5de368b4f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:40.041848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:40.053695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1183/2372] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:40.055603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbb63f-8326-41f2-bda7-b43ac8cc150c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:40.072825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbb63f-8326-41f2-bda7-b43ac8cc150c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:40.084824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad167d-7fb1-47d7-ace7-d28f2d407842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:40.085698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad167d-7fb1-47d7-ace7-d28f2d407842 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:40.086144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad167d-7fb1-47d7-ace7-d28f2d407842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:40.086425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad167d-7fb1-47d7-ace7-d28f2d407842 admin admin] PATH_INFO: `/v3/projects/c605301a5cd5456bbafb025dc67a0cac/users/9dd674904a4141208d9cca76a39dfe66/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:40.108065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad167d-7fb1-47d7-ace7-d28f2d407842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c605301a5cd5456bbafb025dc67a0cac, user_id=9dd674904a4141208d9cca76a39dfe66, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:40.112965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad167d-7fb1-47d7-ace7-d28f2d407842 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:40.135038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1190/2373] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:40 2022] PUT /identity/v3/projects/c605301a5cd5456bbafb025dc67a0cac/users/9dd674904a4141208d9cca76a39dfe66/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:40.141403 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de9336-3ce1-48d1-b8a9-48b5de368b4f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:40.153529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de9336-3ce1-48d1-b8a9-48b5de368b4f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:40.249457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51ddb4-2b0c-4918-bb5e-fa23b2f0e4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:40.250663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51ddb4-2b0c-4918-bb5e-fa23b2f0e4ca admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:40.251149 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51ddb4-2b0c-4918-bb5e-fa23b2f0e4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:40.251534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51ddb4-2b0c-4918-bb5e-fa23b2f0e4ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:40.252595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51ddb4-2b0c-4918-bb5e-fa23b2f0e4ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:40.254757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51ddb4-2b0c-4918-bb5e-fa23b2f0e4ca admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:40.268460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1184/2374] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:40.273895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad167d-7fb1-47d7-ace7-d28f2d407842 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:40.306089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad167d-7fb1-47d7-ace7-d28f2d407842 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:40.306089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de62ce7-e15e-4080-88b9-3d4ac5836940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:40.306089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de62ce7-e15e-4080-88b9-3d4ac5836940 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:40.306089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de62ce7-e15e-4080-88b9-3d4ac5836940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:40.307035 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de62ce7-e15e-4080-88b9-3d4ac5836940 admin admin] PATH_INFO: `/v3/projects/c605301a5cd5456bbafb025dc67a0cac/users/9dd674904a4141208d9cca76a39dfe66/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:40.307035 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de62ce7-e15e-4080-88b9-3d4ac5836940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c605301a5cd5456bbafb025dc67a0cac, user_id=9dd674904a4141208d9cca76a39dfe66, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:40.307703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de62ce7-e15e-4080-88b9-3d4ac5836940 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:40.335692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1191/2375] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:40 2022] PUT /identity/v3/projects/c605301a5cd5456bbafb025dc67a0cac/users/9dd674904a4141208d9cca76a39dfe66/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:41.235489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51ddb4-2b0c-4918-bb5e-fa23b2f0e4ca admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:41.239518 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8e51ddb4-2b0c-4918-bb5e-fa23b2f0e4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:41.287986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51ddb4-2b0c-4918-bb5e-fa23b2f0e4ca admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:41.448200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:41.449618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:41.450212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:41.467060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] PATH_INFO: `/v3/users/e7df0b1f38a549519e64fbd07cd4f319` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:41.477479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7df0b1f38a549519e64fbd07cd4f319)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:41.492543 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:41.740517 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7df0b1f38a549519e64fbd07cd4f319'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:41.777247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7df0b1f38a549519e64fbd07cd4f319'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:41.786302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7df0b1f38a549519e64fbd07cd4f319'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:41.787295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7df0b1f38a549519e64fbd07cd4f319'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:41.853802 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7df0b1f38a549519e64fbd07cd4f319'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:41.898770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1185/2376] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:41 2022] DELETE /identity/v3/users/e7df0b1f38a549519e64fbd07cd4f319 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:41.903854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a65eef-4a4b-4d3b-ae81-96d016fab97f None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:41.905327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a65eef-4a4b-4d3b-ae81-96d016fab97f None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:41.905327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a65eef-4a4b-4d3b-ae81-96d016fab97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:41.942001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-88a65eef-4a4b-4d3b-ae81-96d016fab97f None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:41.943295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-88a65eef-4a4b-4d3b-ae81-96d016fab97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:42.103063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:42.183192 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1192/2377] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:42.207821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:42.209648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:42.251131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:42.355927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0c879-a921-4579-9630-3580d45bce71 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:42.361022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1275ea6-6487-4496-be6d-f5aa2c1c3bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:42.361724 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1275ea6-6487-4496-be6d-f5aa2c1c3bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:42.362005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1275ea6-6487-4496-be6d-f5aa2c1c3bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:42.362465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1275ea6-6487-4496-be6d-f5aa2c1c3bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:42.392233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1275ea6-6487-4496-be6d-f5aa2c1c3bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:42.394506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1275ea6-6487-4496-be6d-f5aa2c1c3bb4 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:42.410507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1186/2378] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:43.595724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a65eef-4a4b-4d3b-ae81-96d016fab97f None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:43.597521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-88a65eef-4a4b-4d3b-ae81-96d016fab97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:43.619486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a65eef-4a4b-4d3b-ae81-96d016fab97f None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:43.635142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff13177-8b9e-4690-be8f-63f949f1c96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:43.638012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff13177-8b9e-4690-be8f-63f949f1c96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:43.639665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff13177-8b9e-4690-be8f-63f949f1c96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:43.640466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff13177-8b9e-4690-be8f-63f949f1c96d admin admin] PATH_INFO: `/v3/projects/43adb894c667464a944b9e6da549c496` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:43.642958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff13177-8b9e-4690-be8f-63f949f1c96d admin admin] RBAC: Authorizing `identity:delete_project(project_id=43adb894c667464a944b9e6da549c496)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:43.647532 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff13177-8b9e-4690-be8f-63f949f1c96d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:43.859339 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-5ff13177-8b9e-4690-be8f-63f949f1c96d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43adb894c667464a944b9e6da549c496', 'request_id': 'req-5ff13177-8b9e-4690-be8f-63f949f1c96d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.906103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1275ea6-6487-4496-be6d-f5aa2c1c3bb4 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:43.907732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e1275ea6-6487-4496-be6d-f5aa2c1c3bb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:43.916911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-5ff13177-8b9e-4690-be8f-63f949f1c96d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43adb894c667464a944b9e6da549c496', 'request_id': 'req-5ff13177-8b9e-4690-be8f-63f949f1c96d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.948938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1275ea6-6487-4496-be6d-f5aa2c1c3bb4 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:43.952442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-5ff13177-8b9e-4690-be8f-63f949f1c96d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43adb894c667464a944b9e6da549c496', 'request_id': 'req-5ff13177-8b9e-4690-be8f-63f949f1c96d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.986016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-5ff13177-8b9e-4690-be8f-63f949f1c96d admin admin] The token cache is being invalidate because project 43adb894c667464a944b9e6da549c496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:43.988024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-5ff13177-8b9e-4690-be8f-63f949f1c96d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.990381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-5ff13177-8b9e-4690-be8f-63f949f1c96d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.998673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1193/2379] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:43 2022] DELETE /identity/v3/projects/43adb894c667464a944b9e6da549c496 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:44.005509 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc58bef-5b08-4758-9d32-c9584d8a6bec None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.009475 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc58bef-5b08-4758-9d32-c9584d8a6bec None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.009692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc58bef-5b08-4758-9d32-c9584d8a6bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.097464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-1bc58bef-5b08-4758-9d32-c9584d8a6bec None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:44.098818 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1bc58bef-5b08-4758-9d32-c9584d8a6bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:44.155986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32314eda-81c8-430e-8965-892924f576c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:44.156768 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32314eda-81c8-430e-8965-892924f576c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.157101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32314eda-81c8-430e-8965-892924f576c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.159114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32314eda-81c8-430e-8965-892924f576c7 admin admin] PATH_INFO: `/v3/users/fc3a7e23a4c04d9fb48d4b3b95091006` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.162595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32314eda-81c8-430e-8965-892924f576c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc3a7e23a4c04d9fb48d4b3b95091006)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:44.166289 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32314eda-81c8-430e-8965-892924f576c7 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:44.324992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-32314eda-81c8-430e-8965-892924f576c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc3a7e23a4c04d9fb48d4b3b95091006'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.352790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1194/2380] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:44.359685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-32314eda-81c8-430e-8965-892924f576c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc3a7e23a4c04d9fb48d4b3b95091006'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.361157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc58bef-5b08-4758-9d32-c9584d8a6bec None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:44.361566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-32314eda-81c8-430e-8965-892924f576c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc3a7e23a4c04d9fb48d4b3b95091006'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.362860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-32314eda-81c8-430e-8965-892924f576c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc3a7e23a4c04d9fb48d4b3b95091006'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.376088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1bc58bef-5b08-4758-9d32-c9584d8a6bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:44.419605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-32314eda-81c8-430e-8965-892924f576c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc3a7e23a4c04d9fb48d4b3b95091006'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.428829 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc58bef-5b08-4758-9d32-c9584d8a6bec None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:44.451736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af9618-558a-40db-846b-17ba30b5e371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:44.452512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af9618-558a-40db-846b-17ba30b5e371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.452904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af9618-558a-40db-846b-17ba30b5e371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.453274 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af9618-558a-40db-846b-17ba30b5e371 admin admin] PATH_INFO: `/v3/users/0ffcf88880dd4c04ae1e80cda44e4908` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.454740 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af9618-558a-40db-846b-17ba30b5e371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ffcf88880dd4c04ae1e80cda44e4908)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:44.457071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af9618-558a-40db-846b-17ba30b5e371 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:44.506465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1187/2381] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:43 2022] DELETE /identity/v3/users/fc3a7e23a4c04d9fb48d4b3b95091006 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:44.506465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e5b3d-b92f-4446-8d12-80ca806e5555 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.506465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e5b3d-b92f-4446-8d12-80ca806e5555 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.506465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e5b3d-b92f-4446-8d12-80ca806e5555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.657362 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-9a2e5b3d-b92f-4446-8d12-80ca806e5555 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:44.658301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-25af9618-558a-40db-846b-17ba30b5e371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ffcf88880dd4c04ae1e80cda44e4908'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.662042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9a2e5b3d-b92f-4446-8d12-80ca806e5555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:44.679238 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:44.694784 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-25af9618-558a-40db-846b-17ba30b5e371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ffcf88880dd4c04ae1e80cda44e4908'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.710077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-25af9618-558a-40db-846b-17ba30b5e371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ffcf88880dd4c04ae1e80cda44e4908'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.730992 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-25af9618-558a-40db-846b-17ba30b5e371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ffcf88880dd4c04ae1e80cda44e4908'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.734884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-25af9618-558a-40db-846b-17ba30b5e371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ffcf88880dd4c04ae1e80cda44e4908'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.751419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1195/2382] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:44 2022] DELETE /identity/v3/users/0ffcf88880dd4c04ae1e80cda44e4908 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:44.757203 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5cb12-9b23-4b2d-9647-7237f7f4dc48 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.758403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5cb12-9b23-4b2d-9647-7237f7f4dc48 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.759391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5cb12-9b23-4b2d-9647-7237f7f4dc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.834649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:44.841665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-eba5cb12-9b23-4b2d-9647-7237f7f4dc48 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:44.843331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-eba5cb12-9b23-4b2d-9647-7237f7f4dc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:44.956066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1196/2383] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:44.956066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d19912-4714-416c-afa1-a76bd19c56a3 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.956066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d19912-4714-416c-afa1-a76bd19c56a3 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.956066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d19912-4714-416c-afa1-a76bd19c56a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.007424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-07d19912-4714-416c-afa1-a76bd19c56a3 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:45.028471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-07d19912-4714-416c-afa1-a76bd19c56a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:45.065872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1197/2384] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:45.068288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d19912-4714-416c-afa1-a76bd19c56a3 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.072240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-07d19912-4714-416c-afa1-a76bd19c56a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:45.075001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1188/2385] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 573 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:45.086595 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e5b3d-b92f-4446-8d12-80ca806e5555 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.135516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e5b3d-b92f-4446-8d12-80ca806e5555 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.166869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d1b8f-d0b2-4cae-a07d-a47da4e274d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.167530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d1b8f-d0b2-4cae-a07d-a47da4e274d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.167818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d1b8f-d0b2-4cae-a07d-a47da4e274d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.172466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d1b8f-d0b2-4cae-a07d-a47da4e274d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.173319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d1b8f-d0b2-4cae-a07d-a47da4e274d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:45.173626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d1b8f-d0b2-4cae-a07d-a47da4e274d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.179819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d1b8f-d0b2-4cae-a07d-a47da4e274d6 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.225394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1189/2386] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:45.226635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d1b8f-d0b2-4cae-a07d-a47da4e274d6 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.228866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8d8d1b8f-d0b2-4cae-a07d-a47da4e274d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:45.329524 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d19912-4714-416c-afa1-a76bd19c56a3 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.348680 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda08082-425b-4663-ac36-cce5daf6f108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.349325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda08082-425b-4663-ac36-cce5daf6f108 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.349604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda08082-425b-4663-ac36-cce5daf6f108 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.349885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda08082-425b-4663-ac36-cce5daf6f108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.389582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda08082-425b-4663-ac36-cce5daf6f108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.399212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda08082-425b-4663-ac36-cce5daf6f108 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.431468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d1b8f-d0b2-4cae-a07d-a47da4e274d6 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.440092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1198/2387] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:45.441522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f518d-e999-4648-bf5f-cb11b8875043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.442270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f518d-e999-4648-bf5f-cb11b8875043 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.442576 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f518d-e999-4648-bf5f-cb11b8875043 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.442769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f518d-e999-4648-bf5f-cb11b8875043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.444465 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda08082-425b-4663-ac36-cce5daf6f108 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.467508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f518d-e999-4648-bf5f-cb11b8875043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.469288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f518d-e999-4648-bf5f-cb11b8875043 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.477891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda08082-425b-4663-ac36-cce5daf6f108 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.518671 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1190/2388] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:45.534108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275adb6-a623-4620-90ca-f7afcc1d83b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.535056 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275adb6-a623-4620-90ca-f7afcc1d83b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.535536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275adb6-a623-4620-90ca-f7afcc1d83b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.535973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275adb6-a623-4620-90ca-f7afcc1d83b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.536856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275adb6-a623-4620-90ca-f7afcc1d83b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.539746 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275adb6-a623-4620-90ca-f7afcc1d83b4 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.625072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1199/2389] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:45 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:45.628434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f518d-e999-4648-bf5f-cb11b8875043 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.652765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f518d-e999-4648-bf5f-cb11b8875043 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.740185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e4f7a-b61d-4f0b-ba2b-6f451416517c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.740892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e4f7a-b61d-4f0b-ba2b-6f451416517c admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.741173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e4f7a-b61d-4f0b-ba2b-6f451416517c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.741439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e4f7a-b61d-4f0b-ba2b-6f451416517c admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.742171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e4f7a-b61d-4f0b-ba2b-6f451416517c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.746836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e4f7a-b61d-4f0b-ba2b-6f451416517c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.800909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1191/2390] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:45 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:45.805534 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275adb6-a623-4620-90ca-f7afcc1d83b4 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.830712 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275adb6-a623-4620-90ca-f7afcc1d83b4 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.840717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2a10b-4f4b-42e0-9afd-ebf780ea7b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.843245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2a10b-4f4b-42e0-9afd-ebf780ea7b27 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.843761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2a10b-4f4b-42e0-9afd-ebf780ea7b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.844394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2a10b-4f4b-42e0-9afd-ebf780ea7b27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.845476 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2a10b-4f4b-42e0-9afd-ebf780ea7b27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.848486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2a10b-4f4b-42e0-9afd-ebf780ea7b27 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.869251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1200/2391] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:45.876567 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e4f7a-b61d-4f0b-ba2b-6f451416517c admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.894136 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e4f7a-b61d-4f0b-ba2b-6f451416517c admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.903154 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cca25a-01c1-450a-8176-bee14d21c4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.907008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cca25a-01c1-450a-8176-bee14d21c4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.907514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cca25a-01c1-450a-8176-bee14d21c4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.907896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cca25a-01c1-450a-8176-bee14d21c4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.908896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cca25a-01c1-450a-8176-bee14d21c4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.910886 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cca25a-01c1-450a-8176-bee14d21c4c2 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.932110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2a10b-4f4b-42e0-9afd-ebf780ea7b27 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.942893 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1192/2392] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:45.952591 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2a10b-4f4b-42e0-9afd-ebf780ea7b27 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.958233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c614c28-6df3-4779-8a6a-50cc9b69667d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.959156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c614c28-6df3-4779-8a6a-50cc9b69667d admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.959638 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c614c28-6df3-4779-8a6a-50cc9b69667d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.960103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c614c28-6df3-4779-8a6a-50cc9b69667d admin admin] PATH_INFO: `/v3/projects/6c7960b468ef461fbe066a791cab7e01/users/9b00c17ab9204a62a931da1688818a03/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.987568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c614c28-6df3-4779-8a6a-50cc9b69667d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c7960b468ef461fbe066a791cab7e01, user_id=9b00c17ab9204a62a931da1688818a03, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.990679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c614c28-6df3-4779-8a6a-50cc9b69667d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:46.020788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1201/2393] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:45 2022] PUT /identity/v3/projects/6c7960b468ef461fbe066a791cab7e01/users/9b00c17ab9204a62a931da1688818a03/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:46.025678 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264b047-0e88-4532-b7a9-b6dc140fb372 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:46.025862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264b047-0e88-4532-b7a9-b6dc140fb372 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:46.026235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264b047-0e88-4532-b7a9-b6dc140fb372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:46.110839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-9264b047-0e88-4532-b7a9-b6dc140fb372 None None] MFA Rules not processed for user `9b00c17ab9204a62a931da1688818a03`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:46.116740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9264b047-0e88-4532-b7a9-b6dc140fb372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:46.132604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c7dc-da43-4d3c-b579-f903631b0c16 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:46.134573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c7dc-da43-4d3c-b579-f903631b0c16 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:46.135490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c7dc-da43-4d3c-b579-f903631b0c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:46.212803 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:46.228530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6030c7dc-da43-4d3c-b579-f903631b0c16 None None] MFA Rules not processed for user `6d5a0cf37e3b4d3084a178f83642c144`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:46.236905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6030c7dc-da43-4d3c-b579-f903631b0c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:46.264217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:46.324536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1193/2394] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:46.335232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264b047-0e88-4532-b7a9-b6dc140fb372 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:46.509721 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1202/2395] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:46.510809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030c7dc-da43-4d3c-b579-f903631b0c16 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:46.514787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6030c7dc-da43-4d3c-b579-f903631b0c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:46.582913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264b047-0e88-4532-b7a9-b6dc140fb372 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:46.617078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03998453-b570-477f-a0d6-03e8283051e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:46.617479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03998453-b570-477f-a0d6-03e8283051e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:46.617958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03998453-b570-477f-a0d6-03e8283051e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:46.618414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03998453-b570-477f-a0d6-03e8283051e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:46.628044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03998453-b570-477f-a0d6-03e8283051e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:46.628044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03998453-b570-477f-a0d6-03e8283051e8 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:46.821403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030c7dc-da43-4d3c-b579-f903631b0c16 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:46.830448 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1194/2396] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:46 2022] POST /identity/v3/projects => generated 383 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:46.835806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03998453-b570-477f-a0d6-03e8283051e8 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:47.053492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7093032-17e0-4416-865e-b621f7a27e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:47.054288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7093032-17e0-4416-865e-b621f7a27e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:47.054594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7093032-17e0-4416-865e-b621f7a27e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:47.054872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7093032-17e0-4416-865e-b621f7a27e99 admin admin] PATH_INFO: `/v3/projects/681cbb7fcb00427c8fca3cfee8718e23` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:47.061077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7093032-17e0-4416-865e-b621f7a27e99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=681cbb7fcb00427c8fca3cfee8718e23)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:47.066022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7093032-17e0-4416-865e-b621f7a27e99 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:47.073873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03998453-b570-477f-a0d6-03e8283051e8 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:47.089011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d304cc-3b22-4072-bdb3-e398e9d494d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:47.090169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d304cc-3b22-4072-bdb3-e398e9d494d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:47.090605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d304cc-3b22-4072-bdb3-e398e9d494d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:47.091764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d304cc-3b22-4072-bdb3-e398e9d494d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:47.093246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d304cc-3b22-4072-bdb3-e398e9d494d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:47.095823 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d304cc-3b22-4072-bdb3-e398e9d494d9 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:47.150215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1195/2397] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:46 2022] POST /identity/v3/users => generated 292 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:47.158967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d304cc-3b22-4072-bdb3-e398e9d494d9 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:47.190183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d304cc-3b22-4072-bdb3-e398e9d494d9 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:47.213660 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c7093032-17e0-4416-865e-b621f7a27e99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681cbb7fcb00427c8fca3cfee8718e23', 'request_id': 'req-c7093032-17e0-4416-865e-b621f7a27e99'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:47.242977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c7093032-17e0-4416-865e-b621f7a27e99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681cbb7fcb00427c8fca3cfee8718e23', 'request_id': 'req-c7093032-17e0-4416-865e-b621f7a27e99'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:47.284224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c7093032-17e0-4416-865e-b621f7a27e99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681cbb7fcb00427c8fca3cfee8718e23', 'request_id': 'req-c7093032-17e0-4416-865e-b621f7a27e99'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:47.291621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:47.369820 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c7093032-17e0-4416-865e-b621f7a27e99 admin admin] The token cache is being invalidate because project 681cbb7fcb00427c8fca3cfee8718e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:47.374551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c7093032-17e0-4416-865e-b621f7a27e99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:47.375865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-c7093032-17e0-4416-865e-b621f7a27e99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:47.378208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1203/2398] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:46 2022] DELETE /identity/v3/projects/681cbb7fcb00427c8fca3cfee8718e23 => generated 0 bytes in 868 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:47.385817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b2839-d73f-40ee-a895-08ab4ad019f0 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:47.386130 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b2839-d73f-40ee-a895-08ab4ad019f0 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:47.386382 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b2839-d73f-40ee-a895-08ab4ad019f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:47.422944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:47.442718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-4f1b2839-d73f-40ee-a895-08ab4ad019f0 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:47.446329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4f1b2839-d73f-40ee-a895-08ab4ad019f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:47.467382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b8784-49f5-4c5d-88a6-aab32222837d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:47.469668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b8784-49f5-4c5d-88a6-aab32222837d admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:47.469946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b8784-49f5-4c5d-88a6-aab32222837d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:47.470281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b8784-49f5-4c5d-88a6-aab32222837d admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:47.477607 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b8784-49f5-4c5d-88a6-aab32222837d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:47.485883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b8784-49f5-4c5d-88a6-aab32222837d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:47.532652 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1196/2399] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:47.537434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b8784-49f5-4c5d-88a6-aab32222837d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:47.547571 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f49b8784-49f5-4c5d-88a6-aab32222837d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:47.684126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1204/2400] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:47.691489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a6782-db86-4047-8248-c1c73ee001ea None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:47.692566 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a6782-db86-4047-8248-c1c73ee001ea None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:47.693380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a6782-db86-4047-8248-c1c73ee001ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:47.707002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b8784-49f5-4c5d-88a6-aab32222837d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:47.732440 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d94fd-c086-49c6-a45f-0b82f64079b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:47.734069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d94fd-c086-49c6-a45f-0b82f64079b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:47.734316 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d94fd-c086-49c6-a45f-0b82f64079b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:47.734594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d94fd-c086-49c6-a45f-0b82f64079b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:47.742640 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d94fd-c086-49c6-a45f-0b82f64079b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:47.756772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d94fd-c086-49c6-a45f-0b82f64079b9 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:47.817487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1197/2401] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:47.819291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d94fd-c086-49c6-a45f-0b82f64079b9 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:47.820932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-f94a6782-db86-4047-8248-c1c73ee001ea None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:47.827344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f94a6782-db86-4047-8248-c1c73ee001ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:47.830457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-cf2d94fd-c086-49c6-a45f-0b82f64079b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:47.870944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1205/2402] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:47.874191 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a6782-db86-4047-8248-c1c73ee001ea None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:47.890975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d94fd-c086-49c6-a45f-0b82f64079b9 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:47.909222 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f473357-62c4-4efd-acb0-c21c38b74a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:47.914159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f473357-62c4-4efd-acb0-c21c38b74a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:47.914159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f473357-62c4-4efd-acb0-c21c38b74a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:47.914159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f473357-62c4-4efd-acb0-c21c38b74a38 admin admin] PATH_INFO: `/v3/projects/6e0ae6dd632e46cc968bdd864a4cb717` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:47.914159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f473357-62c4-4efd-acb0-c21c38b74a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e0ae6dd632e46cc968bdd864a4cb717)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:47.916169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f473357-62c4-4efd-acb0-c21c38b74a38 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:47.924239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a6782-db86-4047-8248-c1c73ee001ea None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:47.934579 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135aa7d2-2949-4631-a7df-7607632b793b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:47.941016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aa7d2-2949-4631-a7df-7607632b793b admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:47.941514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aa7d2-2949-4631-a7df-7607632b793b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:47.942342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aa7d2-2949-4631-a7df-7607632b793b admin admin] PATH_INFO: `/v3/projects/e03b079cc2e04626aea5050f3ee8f36a/users/0024008c73d143698b687718a66eca71/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:47.984453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135aa7d2-2949-4631-a7df-7607632b793b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03b079cc2e04626aea5050f3ee8f36a, user_id=0024008c73d143698b687718a66eca71, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:47.991585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135aa7d2-2949-4631-a7df-7607632b793b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:48.060191 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1206/2403] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:47 2022] PUT /identity/v3/projects/e03b079cc2e04626aea5050f3ee8f36a/users/0024008c73d143698b687718a66eca71/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:48.065645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135aa7d2-2949-4631-a7df-7607632b793b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:48.119279 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8f473357-62c4-4efd-acb0-c21c38b74a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e0ae6dd632e46cc968bdd864a4cb717', 'request_id': 'req-8f473357-62c4-4efd-acb0-c21c38b74a38'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:48.149638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8f473357-62c4-4efd-acb0-c21c38b74a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e0ae6dd632e46cc968bdd864a4cb717', 'request_id': 'req-8f473357-62c4-4efd-acb0-c21c38b74a38'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:48.174922 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8f473357-62c4-4efd-acb0-c21c38b74a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e0ae6dd632e46cc968bdd864a4cb717', 'request_id': 'req-8f473357-62c4-4efd-acb0-c21c38b74a38'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:48.227314 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8f473357-62c4-4efd-acb0-c21c38b74a38 admin admin] The token cache is being invalidate because project 6e0ae6dd632e46cc968bdd864a4cb717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:48.230066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8f473357-62c4-4efd-acb0-c21c38b74a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:48.233868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8f473357-62c4-4efd-acb0-c21c38b74a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:48.240041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1198/2404] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:47 2022] DELETE /identity/v3/projects/6e0ae6dd632e46cc968bdd864a4cb717 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:48.240713 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26054836-9352-469d-88b3-4886b814c933 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:48.246085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26054836-9352-469d-88b3-4886b814c933 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:48.246787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26054836-9352-469d-88b3-4886b814c933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:48.303970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135aa7d2-2949-4631-a7df-7607632b793b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:48.355393 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-26054836-9352-469d-88b3-4886b814c933 None None] MFA Rules not processed for user `0024008c73d143698b687718a66eca71`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:48.355782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-26054836-9352-469d-88b3-4886b814c933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:48.418198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09cd9b-0b6f-483f-be0a-d10974bdeb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:48.419406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09cd9b-0b6f-483f-be0a-d10974bdeb1f admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:48.419947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09cd9b-0b6f-483f-be0a-d10974bdeb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:48.421158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09cd9b-0b6f-483f-be0a-d10974bdeb1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:48.423877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09cd9b-0b6f-483f-be0a-d10974bdeb1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:48.423877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09cd9b-0b6f-483f-be0a-d10974bdeb1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:48.427320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09cd9b-0b6f-483f-be0a-d10974bdeb1f admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:48.457071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1207/2405] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:48.458708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09cd9b-0b6f-483f-be0a-d10974bdeb1f admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:48.462528 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4f09cd9b-0b6f-483f-be0a-d10974bdeb1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:48.479539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:48.487932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09cd9b-0b6f-483f-be0a-d10974bdeb1f admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:48.502139 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:48.503276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:48.503762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:48.506314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] PATH_INFO: `/v3/users/597590bafe384dad83192849b9dfd209` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:48.506541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=597590bafe384dad83192849b9dfd209)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:48.511435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:48.529294 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1199/2406] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:48.529822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26054836-9352-469d-88b3-4886b814c933 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:48.537048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-26054836-9352-469d-88b3-4886b814c933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:48.555397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26054836-9352-469d-88b3-4886b814c933 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:48.566723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634aec6a-a2eb-48f6-9e7d-314fb80c119d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:48.567350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634aec6a-a2eb-48f6-9e7d-314fb80c119d admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:48.567938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634aec6a-a2eb-48f6-9e7d-314fb80c119d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:48.567938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634aec6a-a2eb-48f6-9e7d-314fb80c119d admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:48.568859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634aec6a-a2eb-48f6-9e7d-314fb80c119d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:48.571772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634aec6a-a2eb-48f6-9e7d-314fb80c119d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:48.623727 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '597590bafe384dad83192849b9dfd209'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:48.657773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '597590bafe384dad83192849b9dfd209'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:48.661013 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1200/2407] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:48 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:48.663844 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634aec6a-a2eb-48f6-9e7d-314fb80c119d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:48.666590 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-634aec6a-a2eb-48f6-9e7d-314fb80c119d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:48.685269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '597590bafe384dad83192849b9dfd209'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:48.704437 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '597590bafe384dad83192849b9dfd209'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:48.705502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '597590bafe384dad83192849b9dfd209'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:48.733297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1208/2408] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:48 2022] DELETE /identity/v3/users/597590bafe384dad83192849b9dfd209 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:48.753492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:48.765515 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:48.802292 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:48.949550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634aec6a-a2eb-48f6-9e7d-314fb80c119d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:49.007062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d6a45-269d-400d-9990-a2b4dac18131 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:49.026258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9c736-ab70-43d0-bda9-fd9408b96cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:49.027317 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9c736-ab70-43d0-bda9-fd9408b96cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:49.032493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9c736-ab70-43d0-bda9-fd9408b96cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:49.032971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9c736-ab70-43d0-bda9-fd9408b96cf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:49.033778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9c736-ab70-43d0-bda9-fd9408b96cf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:49.037601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9c736-ab70-43d0-bda9-fd9408b96cf2 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:49.083565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac4d6b-b500-45d2-8313-a99c461692d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:49.084545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac4d6b-b500-45d2-8313-a99c461692d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:49.084913 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac4d6b-b500-45d2-8313-a99c461692d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:49.085393 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac4d6b-b500-45d2-8313-a99c461692d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:49.086579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac4d6b-b500-45d2-8313-a99c461692d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:49.088625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac4d6b-b500-45d2-8313-a99c461692d7 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:49.142278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1209/2409] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:48 2022] POST /identity/v3/users => generated 290 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:49.154585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9c736-ab70-43d0-bda9-fd9408b96cf2 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:49.187351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9c736-ab70-43d0-bda9-fd9408b96cf2 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:49.189508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1201/2410] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:48 2022] POST /identity/v3/projects => generated 383 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:49.193616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac4d6b-b500-45d2-8313-a99c461692d7 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:49.197868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-32ac4d6b-b500-45d2-8313-a99c461692d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:49.352980 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee679a0d-43f9-4936-ac98-740df699094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:49.353383 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac4d6b-b500-45d2-8313-a99c461692d7 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:49.355181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee679a0d-43f9-4936-ac98-740df699094d admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:49.355447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee679a0d-43f9-4936-ac98-740df699094d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:49.355730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee679a0d-43f9-4936-ac98-740df699094d admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:49.356845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee679a0d-43f9-4936-ac98-740df699094d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:49.361354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee679a0d-43f9-4936-ac98-740df699094d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:49.367932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003f29e-a94c-4e66-9475-ff718a52619c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:49.368978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003f29e-a94c-4e66-9475-ff718a52619c admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:49.369267 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003f29e-a94c-4e66-9475-ff718a52619c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:49.369556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003f29e-a94c-4e66-9475-ff718a52619c admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:49.370508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003f29e-a94c-4e66-9475-ff718a52619c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:49.377008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003f29e-a94c-4e66-9475-ff718a52619c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:49.395164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1210/2411] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:49.395700 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee679a0d-43f9-4936-ac98-740df699094d admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:49.408160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:49.417049 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee679a0d-43f9-4936-ac98-740df699094d admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:49.438070 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be7d34-f4b1-4c30-8561-86d97efbd5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:49.438893 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be7d34-f4b1-4c30-8561-86d97efbd5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:49.439301 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be7d34-f4b1-4c30-8561-86d97efbd5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:49.439636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be7d34-f4b1-4c30-8561-86d97efbd5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:49.440446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be7d34-f4b1-4c30-8561-86d97efbd5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:49.442717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be7d34-f4b1-4c30-8561-86d97efbd5d5 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:49.463047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1202/2412] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:49 2022] POST /identity/v3/users => generated 298 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:49.463932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1211/2413] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:49.464129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003f29e-a94c-4e66-9475-ff718a52619c admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:49.468692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be7d34-f4b1-4c30-8561-86d97efbd5d5 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:49.484878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003f29e-a94c-4e66-9475-ff718a52619c admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:49.492541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be7d34-f4b1-4c30-8561-86d97efbd5d5 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:49.503561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9e92b-65d7-4d1b-a28c-c17accd88774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:49.506363 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e92b-65d7-4d1b-a28c-c17accd88774 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:49.507073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e92b-65d7-4d1b-a28c-c17accd88774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:49.507872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e92b-65d7-4d1b-a28c-c17accd88774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:49.509068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9e92b-65d7-4d1b-a28c-c17accd88774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:49.514778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9e92b-65d7-4d1b-a28c-c17accd88774 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:49.519756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b2793-1ab0-4721-b0bb-035f8c8dc9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:49.523432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b2793-1ab0-4721-b0bb-035f8c8dc9a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:49.523730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b2793-1ab0-4721-b0bb-035f8c8dc9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:49.524034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b2793-1ab0-4721-b0bb-035f8c8dc9a6 admin admin] PATH_INFO: `/v3/projects/20cdcff29a0448dc92460b2ddd45617b/users/2839103f52324a1ab1ebd43fe230addc/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:49.545924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1212/2414] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:49.547460 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9e92b-65d7-4d1b-a28c-c17accd88774 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:49.569086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9e92b-65d7-4d1b-a28c-c17accd88774 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:49.592861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b2793-1ab0-4721-b0bb-035f8c8dc9a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20cdcff29a0448dc92460b2ddd45617b, user_id=2839103f52324a1ab1ebd43fe230addc, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:49.603415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b2793-1ab0-4721-b0bb-035f8c8dc9a6 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:49.605640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c277797-4127-4e03-9740-4e0db3bb016e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:49.610498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c277797-4127-4e03-9740-4e0db3bb016e admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:49.611179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c277797-4127-4e03-9740-4e0db3bb016e admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:49.613370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c277797-4127-4e03-9740-4e0db3bb016e admin admin] PATH_INFO: `/v3/projects/a4709278e71a4e1fadde5e4d227941fd/users/2090fb8c56f44ba8921d96895f010e44/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:49.640042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1203/2415] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:49 2022] PUT /identity/v3/projects/20cdcff29a0448dc92460b2ddd45617b/users/2839103f52324a1ab1ebd43fe230addc/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:49.650699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d292c08-70d1-4b69-a925-4df9f990a7a2 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:49.650862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d292c08-70d1-4b69-a925-4df9f990a7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:49.651140 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d292c08-70d1-4b69-a925-4df9f990a7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:49.695757 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c277797-4127-4e03-9740-4e0db3bb016e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4709278e71a4e1fadde5e4d227941fd, user_id=2090fb8c56f44ba8921d96895f010e44, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:49.704659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c277797-4127-4e03-9740-4e0db3bb016e admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:49.742178 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1213/2416] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:49 2022] PUT /identity/v3/projects/a4709278e71a4e1fadde5e4d227941fd/users/2090fb8c56f44ba8921d96895f010e44/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:49.742718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-1d292c08-70d1-4b69-a925-4df9f990a7a2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:49.743653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c277797-4127-4e03-9740-4e0db3bb016e admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:49.751438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1d292c08-70d1-4b69-a925-4df9f990a7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:49.852531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:49.903372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c277797-4127-4e03-9740-4e0db3bb016e admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:49.915428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b682d93-0fb9-4be5-8172-006807aeef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:49.915428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b682d93-0fb9-4be5-8172-006807aeef73 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:49.915428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b682d93-0fb9-4be5-8172-006807aeef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:49.915428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b682d93-0fb9-4be5-8172-006807aeef73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:49.915964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b682d93-0fb9-4be5-8172-006807aeef73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:49.917925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b682d93-0fb9-4be5-8172-006807aeef73 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:49.944849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1214/2417] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:49.949591 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b682d93-0fb9-4be5-8172-006807aeef73 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:49.978281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b682d93-0fb9-4be5-8172-006807aeef73 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:50.015293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1204/2418] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:50.022144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c52432-05ff-4ca5-996e-50b6b91d41ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:50.024164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c52432-05ff-4ca5-996e-50b6b91d41ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:50.024924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c52432-05ff-4ca5-996e-50b6b91d41ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:50.025683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c52432-05ff-4ca5-996e-50b6b91d41ff admin admin] PATH_INFO: `/v3/projects/a4709278e71a4e1fadde5e4d227941fd/users/2090fb8c56f44ba8921d96895f010e44/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:50.033190 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c52432-05ff-4ca5-996e-50b6b91d41ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4709278e71a4e1fadde5e4d227941fd, user_id=2090fb8c56f44ba8921d96895f010e44, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:50.039317 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c52432-05ff-4ca5-996e-50b6b91d41ff admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:50.081348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d292c08-70d1-4b69-a925-4df9f990a7a2 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:50.092252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1d292c08-70d1-4b69-a925-4df9f990a7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:50.112526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1215/2419] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:49 2022] PUT /identity/v3/projects/a4709278e71a4e1fadde5e4d227941fd/users/2090fb8c56f44ba8921d96895f010e44/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:50.121515 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a21cd-8e22-4cea-a251-8d97b3baab50 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:50.125500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a21cd-8e22-4cea-a251-8d97b3baab50 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:50.126079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a21cd-8e22-4cea-a251-8d97b3baab50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:50.268954 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-501a21cd-8e22-4cea-a251-8d97b3baab50 None None] MFA Rules not processed for user `2090fb8c56f44ba8921d96895f010e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:50.271100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-501a21cd-8e22-4cea-a251-8d97b3baab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:50.300791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d292c08-70d1-4b69-a925-4df9f990a7a2 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:50.401526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0eee07-8503-4712-87b7-97d7b8c4c7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:50.402924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eee07-8503-4712-87b7-97d7b8c4c7ad service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:50.402924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eee07-8503-4712-87b7-97d7b8c4c7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:50.404005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eee07-8503-4712-87b7-97d7b8c4c7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:50.511646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0eee07-8503-4712-87b7-97d7b8c4c7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:50.513410 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0eee07-8503-4712-87b7-97d7b8c4c7ad service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:50.524889 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1216/2420] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:50.528186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a21cd-8e22-4cea-a251-8d97b3baab50 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:50.532527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-501a21cd-8e22-4cea-a251-8d97b3baab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:50.535169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1205/2421] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:50.538080 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0eee07-8503-4712-87b7-97d7b8c4c7ad service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:50.538080 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8e0eee07-8503-4712-87b7-97d7b8c4c7ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:50.573605 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a21cd-8e22-4cea-a251-8d97b3baab50 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:50.580379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:50.584308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:50.585685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:50.585685 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] PATH_INFO: `/v3/projects/6da0f07d4605434daf5eb09a872f84b8` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:50.586896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6da0f07d4605434daf5eb09a872f84b8)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:50.589888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:50.610266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:50.625929 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:50.656197 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0eee07-8503-4712-87b7-97d7b8c4c7ad service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:50.687397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6da0f07d4605434daf5eb09a872f84b8', 'request_id': 'req-f132ae08-01de-4676-b2c6-c1b893a327e3'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:50.710834 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6da0f07d4605434daf5eb09a872f84b8', 'request_id': 'req-f132ae08-01de-4676-b2c6-c1b893a327e3'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:50.721966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6da0f07d4605434daf5eb09a872f84b8', 'request_id': 'req-f132ae08-01de-4676-b2c6-c1b893a327e3'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:50.751471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] The token cache is being invalidate because project 6da0f07d4605434daf5eb09a872f84b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:50.751471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:50.751471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:50.751471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1217/2422] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:50 2022] DELETE /identity/v3/projects/6da0f07d4605434daf5eb09a872f84b8 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:50.753797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:50.759109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:50.816428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cf383-8a09-49f2-8e51-30018f3c29ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:50.820972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cf383-8a09-49f2-8e51-30018f3c29ca service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:50.821880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cf383-8a09-49f2-8e51-30018f3c29ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:50.822626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cf383-8a09-49f2-8e51-30018f3c29ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:50.827263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-fa9cf383-8a09-49f2-8e51-30018f3c29ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:50.971388 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cf383-8a09-49f2-8e51-30018f3c29ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:50.974142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cf383-8a09-49f2-8e51-30018f3c29ca service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:50.977859 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132ae08-01de-4676-b2c6-c1b893a327e3 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:50.996427 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1206/2423] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:51.004092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:51.005545 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:51.006040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:51.007112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] PATH_INFO: `/v3/users/431f7d0f412b4d768f720b7b532b98c4` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:51.012168 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=431f7d0f412b4d768f720b7b532b98c4)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:51.016589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:51.167446 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '431f7d0f412b4d768f720b7b532b98c4'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:51.233379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '431f7d0f412b4d768f720b7b532b98c4'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:51.250625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '431f7d0f412b4d768f720b7b532b98c4'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:51.266995 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '431f7d0f412b4d768f720b7b532b98c4'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:51.269030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '431f7d0f412b4d768f720b7b532b98c4'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:51.303874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1218/2424] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:50 2022] DELETE /identity/v3/users/431f7d0f412b4d768f720b7b532b98c4 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:52.855565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cf383-8a09-49f2-8e51-30018f3c29ca service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:52.924408 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cf383-8a09-49f2-8e51-30018f3c29ca service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:53.209226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47729bb8-ec96-4b48-9d72-e19c5402ff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:53.210401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47729bb8-ec96-4b48-9d72-e19c5402ff4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:53.211485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47729bb8-ec96-4b48-9d72-e19c5402ff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:53.217498 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47729bb8-ec96-4b48-9d72-e19c5402ff4e admin admin] PATH_INFO: `/v3/projects/62571f2ead874cc6bcb08054e3888aba` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:53.222940 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47729bb8-ec96-4b48-9d72-e19c5402ff4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=62571f2ead874cc6bcb08054e3888aba)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:53.227196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47729bb8-ec96-4b48-9d72-e19c5402ff4e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:53.399421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:53.410057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-47729bb8-ec96-4b48-9d72-e19c5402ff4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62571f2ead874cc6bcb08054e3888aba', 'request_id': 'req-47729bb8-ec96-4b48-9d72-e19c5402ff4e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:53.444760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-47729bb8-ec96-4b48-9d72-e19c5402ff4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62571f2ead874cc6bcb08054e3888aba', 'request_id': 'req-47729bb8-ec96-4b48-9d72-e19c5402ff4e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:53.445636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:53.448762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:53.475386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-47729bb8-ec96-4b48-9d72-e19c5402ff4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62571f2ead874cc6bcb08054e3888aba', 'request_id': 'req-47729bb8-ec96-4b48-9d72-e19c5402ff4e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:53.532287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-47729bb8-ec96-4b48-9d72-e19c5402ff4e admin admin] The token cache is being invalidate because project 62571f2ead874cc6bcb08054e3888aba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:53.536080 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-47729bb8-ec96-4b48-9d72-e19c5402ff4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:53.538618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-47729bb8-ec96-4b48-9d72-e19c5402ff4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:53.546887 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1207/2425] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:52 2022] DELETE /identity/v3/projects/62571f2ead874cc6bcb08054e3888aba => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:53.556658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e966c-315b-4a4a-a498-6a8973cf3ac9 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:53.557107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e966c-315b-4a4a-a498-6a8973cf3ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:53.557549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e966c-315b-4a4a-a498-6a8973cf3ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:53.645430 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:53.661896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-ae0e966c-315b-4a4a-a498-6a8973cf3ac9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:53.671893 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ae0e966c-315b-4a4a-a498-6a8973cf3ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:53.771844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86200d-5f6a-46af-b3b3-16ec7a37cdf9 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:53.908411 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea66ed-044b-4250-958d-6c2be3cb220a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:53.909339 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea66ed-044b-4250-958d-6c2be3cb220a service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:53.909617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea66ed-044b-4250-958d-6c2be3cb220a service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:53.909986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea66ed-044b-4250-958d-6c2be3cb220a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:53.917967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-13ea66ed-044b-4250-958d-6c2be3cb220a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:53.920229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1208/2426] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:53.930126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e560f6-9bd4-4285-aeb5-de02659eeeb9 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:53.930413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e560f6-9bd4-4285-aeb5-de02659eeeb9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:53.930657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e560f6-9bd4-4285-aeb5-de02659eeeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:53.977690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-02e560f6-9bd4-4285-aeb5-de02659eeeb9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:53.981103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-02e560f6-9bd4-4285-aeb5-de02659eeeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:54.006091 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1209/2427] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:54.012044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e560f6-9bd4-4285-aeb5-de02659eeeb9 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:54.056258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e560f6-9bd4-4285-aeb5-de02659eeeb9 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:54.092648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f14d30-0af1-4755-aeec-4166a8703df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:54.093275 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f14d30-0af1-4755-aeec-4166a8703df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:54.093544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f14d30-0af1-4755-aeec-4166a8703df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:54.093806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f14d30-0af1-4755-aeec-4166a8703df1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:54.094459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f14d30-0af1-4755-aeec-4166a8703df1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:54.094757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f14d30-0af1-4755-aeec-4166a8703df1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:54.100417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f14d30-0af1-4755-aeec-4166a8703df1 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:54.134486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea66ed-044b-4250-958d-6c2be3cb220a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:54.136086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea66ed-044b-4250-958d-6c2be3cb220a service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:54.145827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1210/2428] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:54.150026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f14d30-0af1-4755-aeec-4166a8703df1 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:54.169794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1219/2429] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:54.176949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f14d30-0af1-4755-aeec-4166a8703df1 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:54.182550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102be32-6ffd-4f35-88ad-cda734e4343b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:54.183379 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102be32-6ffd-4f35-88ad-cda734e4343b admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:54.183805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102be32-6ffd-4f35-88ad-cda734e4343b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:54.184258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102be32-6ffd-4f35-88ad-cda734e4343b admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:54.185044 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102be32-6ffd-4f35-88ad-cda734e4343b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:54.187087 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102be32-6ffd-4f35-88ad-cda734e4343b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:54.236513 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1211/2430] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:54 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:54.241041 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea66ed-044b-4250-958d-6c2be3cb220a service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:54.257204 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea66ed-044b-4250-958d-6c2be3cb220a service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:54.396485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39600d-0699-4bb4-9b9a-387d87adfcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:54.398660 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39600d-0699-4bb4-9b9a-387d87adfcd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:54.398833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39600d-0699-4bb4-9b9a-387d87adfcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:54.401214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39600d-0699-4bb4-9b9a-387d87adfcd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:54.404055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39600d-0699-4bb4-9b9a-387d87adfcd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:54.408812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39600d-0699-4bb4-9b9a-387d87adfcd9 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:54.459069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1220/2431] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:54 2022] POST /identity/v3/users => generated 291 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:54.463303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102be32-6ffd-4f35-88ad-cda734e4343b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:54.479065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102be32-6ffd-4f35-88ad-cda734e4343b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:54.490458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942f7d4-63bb-48d3-92ce-d88319ca6ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:54.491147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942f7d4-63bb-48d3-92ce-d88319ca6ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:54.491425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942f7d4-63bb-48d3-92ce-d88319ca6ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:54.491689 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942f7d4-63bb-48d3-92ce-d88319ca6ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:54.492389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942f7d4-63bb-48d3-92ce-d88319ca6ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:54.495094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942f7d4-63bb-48d3-92ce-d88319ca6ee2 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:54.522292 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1212/2432] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:54.533258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942f7d4-63bb-48d3-92ce-d88319ca6ee2 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:54.533470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36f7ef-0ad7-4b50-980b-24addc90b57a None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:54.533653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36f7ef-0ad7-4b50-980b-24addc90b57a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:54.533933 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36f7ef-0ad7-4b50-980b-24addc90b57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:54.551329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942f7d4-63bb-48d3-92ce-d88319ca6ee2 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:54.565984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb0833-9015-4d52-a274-45751801fb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:54.566747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb0833-9015-4d52-a274-45751801fb03 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:54.567053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb0833-9015-4d52-a274-45751801fb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:54.567320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb0833-9015-4d52-a274-45751801fb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:54.568798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb0833-9015-4d52-a274-45751801fb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:54.576228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb0833-9015-4d52-a274-45751801fb03 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:54.607812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1213/2433] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:54.616598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb0833-9015-4d52-a274-45751801fb03 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:54.625035 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7b36f7ef-0ad7-4b50-980b-24addc90b57a None None] MFA Rules not processed for user `9dd674904a4141208d9cca76a39dfe66`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:54.637616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7b36f7ef-0ad7-4b50-980b-24addc90b57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:54.644773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb0833-9015-4d52-a274-45751801fb03 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:54.654750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8539234-ca76-4699-9b44-2270ad0f21f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:54.655444 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8539234-ca76-4699-9b44-2270ad0f21f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:54.655726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8539234-ca76-4699-9b44-2270ad0f21f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:54.656090 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8539234-ca76-4699-9b44-2270ad0f21f9 admin admin] PATH_INFO: `/v3/projects/e4f6fcab728e404d925f9fea14804ad5/users/9b26a12bd7484a9ba288002adc21d071/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:54.685929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8539234-ca76-4699-9b44-2270ad0f21f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f6fcab728e404d925f9fea14804ad5, user_id=9b26a12bd7484a9ba288002adc21d071, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:54.694526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8539234-ca76-4699-9b44-2270ad0f21f9 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:54.717070 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:54.735950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1214/2434] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:54 2022] PUT /identity/v3/projects/e4f6fcab728e404d925f9fea14804ad5/users/9b26a12bd7484a9ba288002adc21d071/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:54.749762 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb215dbc-dcec-43cf-9299-41a30cdb2d62 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:54.750191 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb215dbc-dcec-43cf-9299-41a30cdb2d62 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:54.750355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb215dbc-dcec-43cf-9299-41a30cdb2d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:54.852263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-eb215dbc-dcec-43cf-9299-41a30cdb2d62 None None] MFA Rules not processed for user `9b26a12bd7484a9ba288002adc21d071`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:54.856507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-eb215dbc-dcec-43cf-9299-41a30cdb2d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:54.949843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1221/2435] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:54.959515 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:54.978575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36f7ef-0ad7-4b50-980b-24addc90b57a None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:54.981590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7b36f7ef-0ad7-4b50-980b-24addc90b57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:55.109223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1215/2436] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:55.110866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb215dbc-dcec-43cf-9299-41a30cdb2d62 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:55.276061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36f7ef-0ad7-4b50-980b-24addc90b57a None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:55.372603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb215dbc-dcec-43cf-9299-41a30cdb2d62 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:55.381780 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaf18a-9af1-46e4-a7b2-f3e03084e925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:55.382560 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaf18a-9af1-46e4-a7b2-f3e03084e925 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:55.384073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaf18a-9af1-46e4-a7b2-f3e03084e925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:55.384073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaf18a-9af1-46e4-a7b2-f3e03084e925 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:55.384607 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eaf18a-9af1-46e4-a7b2-f3e03084e925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:55.387255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eaf18a-9af1-46e4-a7b2-f3e03084e925 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:55.464491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1216/2437] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:55 2022] POST /identity/v3/projects => generated 379 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:55.466858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaf18a-9af1-46e4-a7b2-f3e03084e925 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:55.470740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c7eaf18a-9af1-46e4-a7b2-f3e03084e925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:55.476537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd40c7-d1ee-4942-95ab-b82cd49703a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:55.477243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd40c7-d1ee-4942-95ab-b82cd49703a3 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:55.477716 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd40c7-d1ee-4942-95ab-b82cd49703a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:55.477999 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd40c7-d1ee-4942-95ab-b82cd49703a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:55.479871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8abd40c7-d1ee-4942-95ab-b82cd49703a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:55.582772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:55.606798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaf18a-9af1-46e4-a7b2-f3e03084e925 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:55.616428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8976b38-9490-49d1-a0b7-dada553023e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:55.617332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8976b38-9490-49d1-a0b7-dada553023e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:55.617571 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8976b38-9490-49d1-a0b7-dada553023e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:55.618505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8976b38-9490-49d1-a0b7-dada553023e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:55.621263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd40c7-d1ee-4942-95ab-b82cd49703a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:55.623527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd40c7-d1ee-4942-95ab-b82cd49703a3 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:55.624290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e8976b38-9490-49d1-a0b7-dada553023e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:55.666713 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1222/2438] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:55.667703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd40c7-d1ee-4942-95ab-b82cd49703a3 service nova] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:55.732385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd40c7-d1ee-4942-95ab-b82cd49703a3 service nova] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:55.759810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8976b38-9490-49d1-a0b7-dada553023e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:55.763797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8976b38-9490-49d1-a0b7-dada553023e2 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:55.807076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1217/2439] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:55.931770 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f76ec0-03ea-47f2-9856-214ae3fc691a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:55.932612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f76ec0-03ea-47f2-9856-214ae3fc691a admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:55.932892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f76ec0-03ea-47f2-9856-214ae3fc691a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:55.933163 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f76ec0-03ea-47f2-9856-214ae3fc691a admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:55.934523 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f76ec0-03ea-47f2-9856-214ae3fc691a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:55.936639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f76ec0-03ea-47f2-9856-214ae3fc691a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:55.978763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1223/2440] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:55 2022] POST /identity/v3/users => generated 296 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:55.984418 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8976b38-9490-49d1-a0b7-dada553023e2 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:55.998006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8976b38-9490-49d1-a0b7-dada553023e2 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.004777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ba792-304d-4d49-8828-25a53ce86924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.005920 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ba792-304d-4d49-8828-25a53ce86924 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.006359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ba792-304d-4d49-8828-25a53ce86924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.006823 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ba792-304d-4d49-8828-25a53ce86924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.007680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ba792-304d-4d49-8828-25a53ce86924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.011425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ba792-304d-4d49-8828-25a53ce86924 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.017006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:56.031337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1218/2441] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:56.035808 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f76ec0-03ea-47f2-9856-214ae3fc691a admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.070587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f76ec0-03ea-47f2-9856-214ae3fc691a admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.077826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d0e31-e2bb-44b5-aed1-1b3f4f510534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.079146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d0e31-e2bb-44b5-aed1-1b3f4f510534 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.079497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d0e31-e2bb-44b5-aed1-1b3f4f510534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.079771 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d0e31-e2bb-44b5-aed1-1b3f4f510534 admin admin] PATH_INFO: `/v3/projects/494b5edce2bb49108f33c2c25462d914/users/5a2ae9cbef4d494fbdb7b45e4882ea86/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.109127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d0e31-e2bb-44b5-aed1-1b3f4f510534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=494b5edce2bb49108f33c2c25462d914, user_id=5a2ae9cbef4d494fbdb7b45e4882ea86, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.111940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d0e31-e2bb-44b5-aed1-1b3f4f510534 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.141468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1224/2442] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:56 2022] PUT /identity/v3/projects/494b5edce2bb49108f33c2c25462d914/users/5a2ae9cbef4d494fbdb7b45e4882ea86/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:56.142064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ba792-304d-4d49-8828-25a53ce86924 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.161263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ba792-304d-4d49-8828-25a53ce86924 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.325217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e52402-e88d-41a0-bd68-23b3e6f16631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.325217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e52402-e88d-41a0-bd68-23b3e6f16631 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.325217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e52402-e88d-41a0-bd68-23b3e6f16631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.325217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e52402-e88d-41a0-bd68-23b3e6f16631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.327177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e52402-e88d-41a0-bd68-23b3e6f16631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.327689 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e52402-e88d-41a0-bd68-23b3e6f16631 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.368046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1219/2443] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:56.369791 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d0e31-e2bb-44b5-aed1-1b3f4f510534 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.386979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d0e31-e2bb-44b5-aed1-1b3f4f510534 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.408148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713f108-1c87-4701-82aa-a8f2c6182917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.408967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713f108-1c87-4701-82aa-a8f2c6182917 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.409311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713f108-1c87-4701-82aa-a8f2c6182917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.409644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713f108-1c87-4701-82aa-a8f2c6182917 admin admin] PATH_INFO: `/v3/projects/494b5edce2bb49108f33c2c25462d914/users/5a2ae9cbef4d494fbdb7b45e4882ea86/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.414746 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713f108-1c87-4701-82aa-a8f2c6182917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=494b5edce2bb49108f33c2c25462d914, user_id=5a2ae9cbef4d494fbdb7b45e4882ea86, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.423014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713f108-1c87-4701-82aa-a8f2c6182917 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.467572 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1225/2444] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:56 2022] PUT /identity/v3/projects/494b5edce2bb49108f33c2c25462d914/users/5a2ae9cbef4d494fbdb7b45e4882ea86/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:56.469286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174a007-3a53-45be-9803-adf3b248ec24 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.469880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174a007-3a53-45be-9803-adf3b248ec24 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.470337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174a007-3a53-45be-9803-adf3b248ec24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.545124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-2174a007-3a53-45be-9803-adf3b248ec24 None None] MFA Rules not processed for user `5a2ae9cbef4d494fbdb7b45e4882ea86`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:56.551538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2174a007-3a53-45be-9803-adf3b248ec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:56.744861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1220/2445] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:56.757053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713f108-1c87-4701-82aa-a8f2c6182917 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.759185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3713f108-1c87-4701-82aa-a8f2c6182917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:56.899631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713f108-1c87-4701-82aa-a8f2c6182917 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.905158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cea7f-56ed-46b7-97b2-afa9f13518b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.906269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cea7f-56ed-46b7-97b2-afa9f13518b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.906743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cea7f-56ed-46b7-97b2-afa9f13518b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.907199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cea7f-56ed-46b7-97b2-afa9f13518b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.933023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cea7f-56ed-46b7-97b2-afa9f13518b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.935512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cea7f-56ed-46b7-97b2-afa9f13518b0 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.975458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1226/2446] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:58.369672 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174a007-3a53-45be-9803-adf3b248ec24 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:58.581837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174a007-3a53-45be-9803-adf3b248ec24 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:58.592976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:58.593641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:58.593921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:58.594194 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] PATH_INFO: `/v3/users/9b26a12bd7484a9ba288002adc21d071` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:58.604628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b26a12bd7484a9ba288002adc21d071)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:58.607199 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:58.650528 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:58.719027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b26a12bd7484a9ba288002adc21d071'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:58.736643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b26a12bd7484a9ba288002adc21d071'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:58.737968 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b26a12bd7484a9ba288002adc21d071'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:58.739371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b26a12bd7484a9ba288002adc21d071'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:58.788296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b26a12bd7484a9ba288002adc21d071'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:58.881464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1221/2447] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:58 2022] DELETE /identity/v3/users/9b26a12bd7484a9ba288002adc21d071 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:58.883514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da6a10-1c96-4934-92e0-378ca082c5b6 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:58.886163 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da6a10-1c96-4934-92e0-378ca082c5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:58.886617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da6a10-1c96-4934-92e0-378ca082c5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:58.965329 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-55da6a10-1c96-4934-92e0-378ca082c5b6 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:58.966570 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-55da6a10-1c96-4934-92e0-378ca082c5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:59.114088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:59.193978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1227/2448] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:59.237457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:59.239541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:59.449545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4e99b-18fd-41f5-9f8a-31e4ce59107b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:59.470314 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c722c0-c7d0-45ad-b6c9-795ae4b092bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:59.478034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c722c0-c7d0-45ad-b6c9-795ae4b092bf service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:59.478295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c722c0-c7d0-45ad-b6c9-795ae4b092bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:59.478568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c722c0-c7d0-45ad-b6c9-795ae4b092bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:59.512800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c722c0-c7d0-45ad-b6c9-795ae4b092bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:59.514254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c722c0-c7d0-45ad-b6c9-795ae4b092bf service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:59.560890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1222/2449] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:19:01.372169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da6a10-1c96-4934-92e0-378ca082c5b6 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:01.374580 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-55da6a10-1c96-4934-92e0-378ca082c5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:01.430325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da6a10-1c96-4934-92e0-378ca082c5b6 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:01.462150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146d46c-a2c6-46d4-9977-3f5b80e46acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:01.463413 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146d46c-a2c6-46d4-9977-3f5b80e46acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:01.463657 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146d46c-a2c6-46d4-9977-3f5b80e46acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:01.464180 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146d46c-a2c6-46d4-9977-3f5b80e46acf admin admin] PATH_INFO: `/v3/users/9b00c17ab9204a62a931da1688818a03` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:01.470689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146d46c-a2c6-46d4-9977-3f5b80e46acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b00c17ab9204a62a931da1688818a03)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:01.474630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146d46c-a2c6-46d4-9977-3f5b80e46acf admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:01.615461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6146d46c-a2c6-46d4-9977-3f5b80e46acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b00c17ab9204a62a931da1688818a03'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:01.656556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c722c0-c7d0-45ad-b6c9-795ae4b092bf service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:01.659239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6146d46c-a2c6-46d4-9977-3f5b80e46acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b00c17ab9204a62a931da1688818a03'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:01.661522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e2c722c0-c7d0-45ad-b6c9-795ae4b092bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:01.677801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6146d46c-a2c6-46d4-9977-3f5b80e46acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b00c17ab9204a62a931da1688818a03'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:01.702117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6146d46c-a2c6-46d4-9977-3f5b80e46acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b00c17ab9204a62a931da1688818a03'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:01.702928 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6146d46c-a2c6-46d4-9977-3f5b80e46acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b00c17ab9204a62a931da1688818a03'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:01.711429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1228/2450] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:01 2022] DELETE /identity/v3/users/9b00c17ab9204a62a931da1688818a03 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:01.721491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c722c0-c7d0-45ad-b6c9-795ae4b092bf service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:01.723764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38001a-09a6-4a22-91be-f7c4fd23e60e None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:01.724100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38001a-09a6-4a22-91be-f7c4fd23e60e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:01.724385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38001a-09a6-4a22-91be-f7c4fd23e60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:01.779037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-1b38001a-09a6-4a22-91be-f7c4fd23e60e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:01.781080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1b38001a-09a6-4a22-91be-f7c4fd23e60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:01.949334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1229/2451] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:01.970923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:01.973224 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:01.976956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:01.979782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38001a-09a6-4a22-91be-f7c4fd23e60e None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:01.982071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1b38001a-09a6-4a22-91be-f7c4fd23e60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:01.985708 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] PATH_INFO: `/v3/projects/e4f6fcab728e404d925f9fea14804ad5` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:01.990536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4f6fcab728e404d925f9fea14804ad5)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:02.002143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:02.141165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38001a-09a6-4a22-91be-f7c4fd23e60e None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:02.146459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:02.166790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4f6fcab728e404d925f9fea14804ad5', 'request_id': 'req-43d05c34-25b5-4151-9ea1-9c96a29a565b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.185166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4f6fcab728e404d925f9fea14804ad5', 'request_id': 'req-43d05c34-25b5-4151-9ea1-9c96a29a565b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.198213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:02.220761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4f6fcab728e404d925f9fea14804ad5', 'request_id': 'req-43d05c34-25b5-4151-9ea1-9c96a29a565b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.253838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] The token cache is being invalidate because project e4f6fcab728e404d925f9fea14804ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:02.254441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.255524 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.257765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1223/2452] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:01 2022] DELETE /identity/v3/projects/e4f6fcab728e404d925f9fea14804ad5 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:02.264304 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:02.267058 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:02.357435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83e191-5bfa-4dba-b760-3ec5b12b66c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:02.358786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83e191-5bfa-4dba-b760-3ec5b12b66c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:02.359688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83e191-5bfa-4dba-b760-3ec5b12b66c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:02.364326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83e191-5bfa-4dba-b760-3ec5b12b66c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:02.370768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-de83e191-5bfa-4dba-b760-3ec5b12b66c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:02.525375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d05c34-25b5-4151-9ea1-9c96a29a565b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:02.540919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693768f8-a22f-481e-a344-014372371323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:02.542306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693768f8-a22f-481e-a344-014372371323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:02.543416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693768f8-a22f-481e-a344-014372371323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:02.544017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693768f8-a22f-481e-a344-014372371323 admin admin] PATH_INFO: `/v3/users/5a2ae9cbef4d494fbdb7b45e4882ea86` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:02.554567 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693768f8-a22f-481e-a344-014372371323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2ae9cbef4d494fbdb7b45e4882ea86)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:02.557396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693768f8-a22f-481e-a344-014372371323 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:02.668389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83e191-5bfa-4dba-b760-3ec5b12b66c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:02.669805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83e191-5bfa-4dba-b760-3ec5b12b66c8 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:02.697432 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1230/2453] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:19:02.699015 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d08166-3db1-440b-a4e4-bd0f7ff1ea7b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:02.699344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d08166-3db1-440b-a4e4-bd0f7ff1ea7b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:02.699669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d08166-3db1-440b-a4e4-bd0f7ff1ea7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:02.737029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-693768f8-a22f-481e-a344-014372371323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2ae9cbef4d494fbdb7b45e4882ea86'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.750752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-693768f8-a22f-481e-a344-014372371323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2ae9cbef4d494fbdb7b45e4882ea86'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.752270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-693768f8-a22f-481e-a344-014372371323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2ae9cbef4d494fbdb7b45e4882ea86'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.754810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-693768f8-a22f-481e-a344-014372371323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2ae9cbef4d494fbdb7b45e4882ea86'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.767365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-90d08166-3db1-440b-a4e4-bd0f7ff1ea7b None None] MFA Rules not processed for user `e76eba6f76f84c5cb006e079aa86157a`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:02.769290 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-90d08166-3db1-440b-a4e4-bd0f7ff1ea7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:02.791753 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-693768f8-a22f-481e-a344-014372371323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2ae9cbef4d494fbdb7b45e4882ea86'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.834651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1224/2454] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:02 2022] DELETE /identity/v3/users/5a2ae9cbef4d494fbdb7b45e4882ea86 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:02.883967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1231/2455] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:04.052515 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693768f8-a22f-481e-a344-014372371323 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:04.054268 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-693768f8-a22f-481e-a344-014372371323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:04.255316 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d08166-3db1-440b-a4e4-bd0f7ff1ea7b None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:04.257670 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-90d08166-3db1-440b-a4e4-bd0f7ff1ea7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:04.275270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:04.306730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d08166-3db1-440b-a4e4-bd0f7ff1ea7b None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:04.329533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693768f8-a22f-481e-a344-014372371323 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:04.342979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19013a7-5a3d-42a1-997e-7f7de0819e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:04.343770 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19013a7-5a3d-42a1-997e-7f7de0819e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:04.344202 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19013a7-5a3d-42a1-997e-7f7de0819e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:04.344580 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19013a7-5a3d-42a1-997e-7f7de0819e62 admin admin] PATH_INFO: `/v3/projects/494b5edce2bb49108f33c2c25462d914` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:04.346140 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19013a7-5a3d-42a1-997e-7f7de0819e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=494b5edce2bb49108f33c2c25462d914)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:04.349343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19013a7-5a3d-42a1-997e-7f7de0819e62 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:04.349678 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:04.350858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:04.351322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:04.352840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] PATH_INFO: `/v3/projects/6c7960b468ef461fbe066a791cab7e01` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:04.357798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c7960b468ef461fbe066a791cab7e01)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:04.368118 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:04.491987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b19013a7-5a3d-42a1-997e-7f7de0819e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '494b5edce2bb49108f33c2c25462d914', 'request_id': 'req-b19013a7-5a3d-42a1-997e-7f7de0819e62'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.504133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b19013a7-5a3d-42a1-997e-7f7de0819e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '494b5edce2bb49108f33c2c25462d914', 'request_id': 'req-b19013a7-5a3d-42a1-997e-7f7de0819e62'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.513646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:04.515805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b19013a7-5a3d-42a1-997e-7f7de0819e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '494b5edce2bb49108f33c2c25462d914', 'request_id': 'req-b19013a7-5a3d-42a1-997e-7f7de0819e62'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.544185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c7960b468ef461fbe066a791cab7e01', 'request_id': 'req-2233f04f-d497-413e-8526-93eb9684ac41'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.567904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c7960b468ef461fbe066a791cab7e01', 'request_id': 'req-2233f04f-d497-413e-8526-93eb9684ac41'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.590816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b19013a7-5a3d-42a1-997e-7f7de0819e62 admin admin] The token cache is being invalidate because project 494b5edce2bb49108f33c2c25462d914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:04.591173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b19013a7-5a3d-42a1-997e-7f7de0819e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.594508 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b19013a7-5a3d-42a1-997e-7f7de0819e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.596538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1232/2456] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:04 2022] DELETE /identity/v3/projects/494b5edce2bb49108f33c2c25462d914 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:04.597511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c7960b468ef461fbe066a791cab7e01', 'request_id': 'req-2233f04f-d497-413e-8526-93eb9684ac41'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.605352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e0a82-0ae8-4a41-98a3-d4d27f09f339 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:04.605662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e0a82-0ae8-4a41-98a3-d4d27f09f339 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:04.605941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e0a82-0ae8-4a41-98a3-d4d27f09f339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:04.627117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] The token cache is being invalidate because project 6c7960b468ef461fbe066a791cab7e01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:04.627468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.628649 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.633332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1225/2457] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:04 2022] DELETE /identity/v3/projects/6c7960b468ef461fbe066a791cab7e01 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:04.636190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:04.638905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:04.651654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-0d3e0a82-0ae8-4a41-98a3-d4d27f09f339 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:04.653483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0d3e0a82-0ae8-4a41-98a3-d4d27f09f339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:04.823724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1233/2458] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:04.831416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233f04f-d497-413e-8526-93eb9684ac41 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:04.835822 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a209a1b-6cb2-4687-9a8e-dc9962c35947 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:04.836291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a209a1b-6cb2-4687-9a8e-dc9962c35947 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:04.837076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a209a1b-6cb2-4687-9a8e-dc9962c35947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:04.837205 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:04.838414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:04.838699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:04.838974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] PATH_INFO: `/v3/users/0024008c73d143698b687718a66eca71` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:04.841083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0024008c73d143698b687718a66eca71)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:04.843077 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:04.865043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-1a209a1b-6cb2-4687-9a8e-dc9962c35947 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:04.865043 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1a209a1b-6cb2-4687-9a8e-dc9962c35947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:04.875421 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1234/2459] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:04.882479 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a209a1b-6cb2-4687-9a8e-dc9962c35947 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:04.899494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a209a1b-6cb2-4687-9a8e-dc9962c35947 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:04.910549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19574ba9-47b5-4b9e-bc4d-fdc38956073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:04.911156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19574ba9-47b5-4b9e-bc4d-fdc38956073c admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:04.911440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19574ba9-47b5-4b9e-bc4d-fdc38956073c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:04.911710 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19574ba9-47b5-4b9e-bc4d-fdc38956073c admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:04.912428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19574ba9-47b5-4b9e-bc4d-fdc38956073c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:04.912734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19574ba9-47b5-4b9e-bc4d-fdc38956073c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:04.916149 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19574ba9-47b5-4b9e-bc4d-fdc38956073c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:04.936069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0024008c73d143698b687718a66eca71'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.951127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1235/2460] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:19:04.958703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19574ba9-47b5-4b9e-bc4d-fdc38956073c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:04.970397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0024008c73d143698b687718a66eca71'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.971890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0024008c73d143698b687718a66eca71'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.973065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0024008c73d143698b687718a66eca71'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.992376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19574ba9-47b5-4b9e-bc4d-fdc38956073c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.014432 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0024008c73d143698b687718a66eca71'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:05.031443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562b0cd-06ea-4f68-ad50-f901fdf773e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.035751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562b0cd-06ea-4f68-ad50-f901fdf773e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.037365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562b0cd-06ea-4f68-ad50-f901fdf773e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.041028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562b0cd-06ea-4f68-ad50-f901fdf773e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.051429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562b0cd-06ea-4f68-ad50-f901fdf773e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.058447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562b0cd-06ea-4f68-ad50-f901fdf773e4 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:05.091745 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:05.109635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1226/2461] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:04 2022] DELETE /identity/v3/users/0024008c73d143698b687718a66eca71 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:05.150889 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1236/2462] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:04 2022] POST /identity/v3/projects => generated 383 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:19:05.157023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:05.160497 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:05.341698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec456c-0788-4bad-b22a-bc42bdbebc5a admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.498771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128af32-671b-4c2d-b9f6-a2de84ac6c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.499793 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128af32-671b-4c2d-b9f6-a2de84ac6c9e admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.500345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128af32-671b-4c2d-b9f6-a2de84ac6c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.502005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128af32-671b-4c2d-b9f6-a2de84ac6c9e admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.504139 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128af32-671b-4c2d-b9f6-a2de84ac6c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.509651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128af32-671b-4c2d-b9f6-a2de84ac6c9e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:05.583265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1227/2463] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:05 2022] POST /identity/v3/users => generated 292 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:05.588324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562b0cd-06ea-4f68-ad50-f901fdf773e4 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:05.630337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562b0cd-06ea-4f68-ad50-f901fdf773e4 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.644844 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeda626-95f1-4ff2-811f-d45bd92ca7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.645650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeda626-95f1-4ff2-811f-d45bd92ca7df admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.645963 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeda626-95f1-4ff2-811f-d45bd92ca7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.647538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeda626-95f1-4ff2-811f-d45bd92ca7df admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.648387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeda626-95f1-4ff2-811f-d45bd92ca7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.650077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeda626-95f1-4ff2-811f-d45bd92ca7df admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:05.672348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1237/2464] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:05.678635 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128af32-671b-4c2d-b9f6-a2de84ac6c9e admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:05.724537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128af32-671b-4c2d-b9f6-a2de84ac6c9e admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.731945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a1ed5-a9c4-4f70-9049-0927e4ccd520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.733341 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a1ed5-a9c4-4f70-9049-0927e4ccd520 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.733781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a1ed5-a9c4-4f70-9049-0927e4ccd520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.734203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a1ed5-a9c4-4f70-9049-0927e4ccd520 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.735029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a1ed5-a9c4-4f70-9049-0927e4ccd520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.736979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a1ed5-a9c4-4f70-9049-0927e4ccd520 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:05.751544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1228/2465] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:19:05.755891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeda626-95f1-4ff2-811f-d45bd92ca7df admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:05.773111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeda626-95f1-4ff2-811f-d45bd92ca7df admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.782279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf3db9-e35f-4892-a4bf-f845a0b62f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.782279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf3db9-e35f-4892-a4bf-f845a0b62f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.782279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf3db9-e35f-4892-a4bf-f845a0b62f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.782279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf3db9-e35f-4892-a4bf-f845a0b62f96 admin admin] PATH_INFO: `/v3/projects/b4aa238cfc3d4806bd775e10a2256f94/users/608b9c6173a64abfa6681131b721e22e/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.805047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf3db9-e35f-4892-a4bf-f845a0b62f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4aa238cfc3d4806bd775e10a2256f94, user_id=608b9c6173a64abfa6681131b721e22e, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.809406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf3db9-e35f-4892-a4bf-f845a0b62f96 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:05.830855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1238/2466] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:05 2022] PUT /identity/v3/projects/b4aa238cfc3d4806bd775e10a2256f94/users/608b9c6173a64abfa6681131b721e22e/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:05.851723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbba907-2ca8-40ee-b290-78972e8256c0 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.851723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbba907-2ca8-40ee-b290-78972e8256c0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.851723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbba907-2ca8-40ee-b290-78972e8256c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.886427 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-6dbba907-2ca8-40ee-b290-78972e8256c0 None None] MFA Rules not processed for user `608b9c6173a64abfa6681131b721e22e`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:05.891250 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6dbba907-2ca8-40ee-b290-78972e8256c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:06.004460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1229/2467] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:06.019120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf3db9-e35f-4892-a4bf-f845a0b62f96 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:06.021071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-becf3db9-e35f-4892-a4bf-f845a0b62f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:06.102456 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf3db9-e35f-4892-a4bf-f845a0b62f96 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:06.108099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e796a4-e587-414e-9b71-0143dbefff6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:06.108804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e796a4-e587-414e-9b71-0143dbefff6b service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:06.109147 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e796a4-e587-414e-9b71-0143dbefff6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:06.109479 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e796a4-e587-414e-9b71-0143dbefff6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:06.122406 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e796a4-e587-414e-9b71-0143dbefff6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:06.125106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e796a4-e587-414e-9b71-0143dbefff6b service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:06.151004 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1239/2468] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:06.206104 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbba907-2ca8-40ee-b290-78972e8256c0 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:06.229728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:06.310367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbba907-2ca8-40ee-b290-78972e8256c0 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:06.315647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012937d5-d139-4d27-a379-e80b37ae6f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:06.316490 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012937d5-d139-4d27-a379-e80b37ae6f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:06.316770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012937d5-d139-4d27-a379-e80b37ae6f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:06.317042 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012937d5-d139-4d27-a379-e80b37ae6f65 admin admin] PATH_INFO: `/v3/users/608b9c6173a64abfa6681131b721e22e` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:06.319691 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012937d5-d139-4d27-a379-e80b37ae6f65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=608b9c6173a64abfa6681131b721e22e)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:06.321661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012937d5-d139-4d27-a379-e80b37ae6f65 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:06.329345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e796a4-e587-414e-9b71-0143dbefff6b service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:06.332920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-29e796a4-e587-414e-9b71-0143dbefff6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:06.362188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e796a4-e587-414e-9b71-0143dbefff6b service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:06.392455 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac931e3-ee35-4d64-8817-76fbf038042c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:06.393495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac931e3-ee35-4d64-8817-76fbf038042c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:06.393958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac931e3-ee35-4d64-8817-76fbf038042c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:06.396496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac931e3-ee35-4d64-8817-76fbf038042c admin admin] PATH_INFO: `/v3/projects/e03b079cc2e04626aea5050f3ee8f36a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:06.401590 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac931e3-ee35-4d64-8817-76fbf038042c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03b079cc2e04626aea5050f3ee8f36a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:06.414540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac931e3-ee35-4d64-8817-76fbf038042c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:06.431030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:06.463469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-012937d5-d139-4d27-a379-e80b37ae6f65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '608b9c6173a64abfa6681131b721e22e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:06.474584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-012937d5-d139-4d27-a379-e80b37ae6f65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '608b9c6173a64abfa6681131b721e22e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:06.475377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-012937d5-d139-4d27-a379-e80b37ae6f65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '608b9c6173a64abfa6681131b721e22e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:06.476073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-012937d5-d139-4d27-a379-e80b37ae6f65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '608b9c6173a64abfa6681131b721e22e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:06.494156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9ac931e3-ee35-4d64-8817-76fbf038042c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03b079cc2e04626aea5050f3ee8f36a', 'request_id': 'req-9ac931e3-ee35-4d64-8817-76fbf038042c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:06.499539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-012937d5-d139-4d27-a379-e80b37ae6f65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '608b9c6173a64abfa6681131b721e22e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:06.511707 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9ac931e3-ee35-4d64-8817-76fbf038042c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03b079cc2e04626aea5050f3ee8f36a', 'request_id': 'req-9ac931e3-ee35-4d64-8817-76fbf038042c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:06.518225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1230/2469] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:06 2022] DELETE /identity/v3/users/608b9c6173a64abfa6681131b721e22e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:06.519840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cf20f-4475-4243-9d57-101c9d4613de None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:06.520229 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cf20f-4475-4243-9d57-101c9d4613de None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:06.520578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754cf20f-4475-4243-9d57-101c9d4613de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:06.534826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9ac931e3-ee35-4d64-8817-76fbf038042c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03b079cc2e04626aea5050f3ee8f36a', 'request_id': 'req-9ac931e3-ee35-4d64-8817-76fbf038042c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:06.582368 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9ac931e3-ee35-4d64-8817-76fbf038042c admin admin] The token cache is being invalidate because project e03b079cc2e04626aea5050f3ee8f36a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:06.582840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9ac931e3-ee35-4d64-8817-76fbf038042c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:06.583015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-754cf20f-4475-4243-9d57-101c9d4613de None None] MFA Rules not processed for user `2839103f52324a1ab1ebd43fe230addc`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:06.583655 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9ac931e3-ee35-4d64-8817-76fbf038042c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:06.584691 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-754cf20f-4475-4243-9d57-101c9d4613de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:06.588441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1240/2470] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:06 2022] DELETE /identity/v3/projects/e03b079cc2e04626aea5050f3ee8f36a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:06.591985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4b58b-2a04-4650-85df-a543c8e27e20 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:06.592270 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4b58b-2a04-4650-85df-a543c8e27e20 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:06.593281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4b58b-2a04-4650-85df-a543c8e27e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:06.615867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-64a4b58b-2a04-4650-85df-a543c8e27e20 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:06.617226 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-64a4b58b-2a04-4650-85df-a543c8e27e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:06.690888 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1231/2471] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:06.694654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cf20f-4475-4243-9d57-101c9d4613de None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:06.697721 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-754cf20f-4475-4243-9d57-101c9d4613de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:06.740994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1241/2472] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:06.742568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4b58b-2a04-4650-85df-a543c8e27e20 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:06.744845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-64a4b58b-2a04-4650-85df-a543c8e27e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:06.769641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4b58b-2a04-4650-85df-a543c8e27e20 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:06.778167 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:06.778668 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1f257-0594-4848-b8dc-bf27c1f93fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:06.779738 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1f257-0594-4848-b8dc-bf27c1f93fae admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:06.780260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1f257-0594-4848-b8dc-bf27c1f93fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:06.780726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1f257-0594-4848-b8dc-bf27c1f93fae admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:06.787417 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1f257-0594-4848-b8dc-bf27c1f93fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:06.791340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1f257-0594-4848-b8dc-bf27c1f93fae admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:06.863399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1242/2473] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:06 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:06.864908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1f257-0594-4848-b8dc-bf27c1f93fae admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:06.867790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-18b1f257-0594-4848-b8dc-bf27c1f93fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:06.878720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754cf20f-4475-4243-9d57-101c9d4613de None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:06.977598 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:06.978172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:06.978633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:06.978914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] PATH_INFO: `/v3/users/2090fb8c56f44ba8921d96895f010e44` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:06.986260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2090fb8c56f44ba8921d96895f010e44)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:06.990963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:07.039821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1f257-0594-4848-b8dc-bf27c1f93fae admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:07.047283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83966ddc-1b91-4e4b-92af-c8753bb0e7a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:07.047961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83966ddc-1b91-4e4b-92af-c8753bb0e7a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:07.048315 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83966ddc-1b91-4e4b-92af-c8753bb0e7a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:07.048543 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83966ddc-1b91-4e4b-92af-c8753bb0e7a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:07.062241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83966ddc-1b91-4e4b-92af-c8753bb0e7a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:07.064318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83966ddc-1b91-4e4b-92af-c8753bb0e7a5 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:07.082994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1243/2474] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:07.085524 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83966ddc-1b91-4e4b-92af-c8753bb0e7a5 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:07.097396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2090fb8c56f44ba8921d96895f010e44'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:07.107071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83966ddc-1b91-4e4b-92af-c8753bb0e7a5 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:07.110533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2090fb8c56f44ba8921d96895f010e44'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:07.111310 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2090fb8c56f44ba8921d96895f010e44'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:07.112290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2090fb8c56f44ba8921d96895f010e44'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:07.121698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e835c2-2c09-4dea-ac00-f220939fd680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:07.122956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e835c2-2c09-4dea-ac00-f220939fd680 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:07.123786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e835c2-2c09-4dea-ac00-f220939fd680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:07.124330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e835c2-2c09-4dea-ac00-f220939fd680 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:07.125669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e835c2-2c09-4dea-ac00-f220939fd680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:07.137454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e835c2-2c09-4dea-ac00-f220939fd680 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:07.142735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2090fb8c56f44ba8921d96895f010e44'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:07.163253 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1232/2475] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:06 2022] DELETE /identity/v3/users/2090fb8c56f44ba8921d96895f010e44 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:07.195796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1244/2476] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:07 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:07.201734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:07.204236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:07.422588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:07.453228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbdf75-c5e0-4b5a-a567-3ab23f5aba3a admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:07.461251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a3bca-d7f8-4f17-89f3-e18d4c597c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:07.466321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a3bca-d7f8-4f17-89f3-e18d4c597c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:07.467002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a3bca-d7f8-4f17-89f3-e18d4c597c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:07.468173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a3bca-d7f8-4f17-89f3-e18d4c597c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:07.472145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a3bca-d7f8-4f17-89f3-e18d4c597c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:07.475841 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a3bca-d7f8-4f17-89f3-e18d4c597c4e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:07.520608 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1233/2477] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:07.521723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e835c2-2c09-4dea-ac00-f220939fd680 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:07.529071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:07.541592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e835c2-2c09-4dea-ac00-f220939fd680 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:07.559442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b6a78-51b9-48ae-abf8-e7b61700dd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:07.561551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b6a78-51b9-48ae-abf8-e7b61700dd5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:07.562117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b6a78-51b9-48ae-abf8-e7b61700dd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:07.562638 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b6a78-51b9-48ae-abf8-e7b61700dd5c admin admin] PATH_INFO: `/v3/projects/c1519945e9154f16adf095a4e25a067d/users/b5398b12577a41cb9891845bc3671f95/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:07.601915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b6a78-51b9-48ae-abf8-e7b61700dd5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1519945e9154f16adf095a4e25a067d, user_id=b5398b12577a41cb9891845bc3671f95, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:07.605848 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b6a78-51b9-48ae-abf8-e7b61700dd5c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:07.641403 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1245/2478] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:07 2022] PUT /identity/v3/projects/c1519945e9154f16adf095a4e25a067d/users/b5398b12577a41cb9891845bc3671f95/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:07.644748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a3bca-d7f8-4f17-89f3-e18d4c597c4e admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:07.672681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a3bca-d7f8-4f17-89f3-e18d4c597c4e admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:07.681769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b6a78-51b9-48ae-abf8-e7b61700dd5c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:07.684405 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-de6b6a78-51b9-48ae-abf8-e7b61700dd5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:07.783529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd72fa8-78af-4bd9-a2ec-cf9a1167f0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:07.784213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd72fa8-78af-4bd9-a2ec-cf9a1167f0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:07.784497 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd72fa8-78af-4bd9-a2ec-cf9a1167f0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:07.784767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd72fa8-78af-4bd9-a2ec-cf9a1167f0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:07.785437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd72fa8-78af-4bd9-a2ec-cf9a1167f0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:07.787306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd72fa8-78af-4bd9-a2ec-cf9a1167f0e6 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:07.814946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1234/2479] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:07.820945 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd72fa8-78af-4bd9-a2ec-cf9a1167f0e6 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:07.842694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd72fa8-78af-4bd9-a2ec-cf9a1167f0e6 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:07.871135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109db40-8832-4291-b501-b3c56294993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:07.872072 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109db40-8832-4291-b501-b3c56294993d admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:07.872520 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109db40-8832-4291-b501-b3c56294993d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:07.872954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109db40-8832-4291-b501-b3c56294993d admin admin] PATH_INFO: `/v3/projects/c1519945e9154f16adf095a4e25a067d/users/b5398b12577a41cb9891845bc3671f95/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:07.875183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b6a78-51b9-48ae-abf8-e7b61700dd5c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:07.877468 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109db40-8832-4291-b501-b3c56294993d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1519945e9154f16adf095a4e25a067d, user_id=b5398b12577a41cb9891845bc3671f95, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:07.881223 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109db40-8832-4291-b501-b3c56294993d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:07.890777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91b04f-5d5b-4977-9f24-20eb554ca369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:07.891489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91b04f-5d5b-4977-9f24-20eb554ca369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:07.891770 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91b04f-5d5b-4977-9f24-20eb554ca369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:07.892073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91b04f-5d5b-4977-9f24-20eb554ca369 admin admin] PATH_INFO: `/v3/projects/b4aa238cfc3d4806bd775e10a2256f94` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:07.893297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91b04f-5d5b-4977-9f24-20eb554ca369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4aa238cfc3d4806bd775e10a2256f94)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:07.895267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91b04f-5d5b-4977-9f24-20eb554ca369 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:07.900546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1235/2480] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:07 2022] PUT /identity/v3/projects/c1519945e9154f16adf095a4e25a067d/users/b5398b12577a41cb9891845bc3671f95/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:07.920892 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109db40-8832-4291-b501-b3c56294993d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:07.923977 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1109db40-8832-4291-b501-b3c56294993d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:08.037688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-da91b04f-5d5b-4977-9f24-20eb554ca369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4aa238cfc3d4806bd775e10a2256f94', 'request_id': 'req-da91b04f-5d5b-4977-9f24-20eb554ca369'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:08.059215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-da91b04f-5d5b-4977-9f24-20eb554ca369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4aa238cfc3d4806bd775e10a2256f94', 'request_id': 'req-da91b04f-5d5b-4977-9f24-20eb554ca369'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:08.082156 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-da91b04f-5d5b-4977-9f24-20eb554ca369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4aa238cfc3d4806bd775e10a2256f94', 'request_id': 'req-da91b04f-5d5b-4977-9f24-20eb554ca369'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:08.107439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-da91b04f-5d5b-4977-9f24-20eb554ca369 admin admin] The token cache is being invalidate because project b4aa238cfc3d4806bd775e10a2256f94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:08.110165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-da91b04f-5d5b-4977-9f24-20eb554ca369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:08.113500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-da91b04f-5d5b-4977-9f24-20eb554ca369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:08.118160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1246/2481] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:07 2022] DELETE /identity/v3/projects/b4aa238cfc3d4806bd775e10a2256f94 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:08.123314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f9dca-cc7d-426f-82a8-2916dc605b83 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:08.123959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f9dca-cc7d-426f-82a8-2916dc605b83 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:08.124450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f9dca-cc7d-426f-82a8-2916dc605b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:08.184800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109db40-8832-4291-b501-b3c56294993d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:08.188913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-cc2f9dca-cc7d-426f-82a8-2916dc605b83 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:08.190636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-cc2f9dca-cc7d-426f-82a8-2916dc605b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:08.302485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1247/2482] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:08.314285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e34af-c3ff-4315-83f7-3812157362ca None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:08.314869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e34af-c3ff-4315-83f7-3812157362ca None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:08.315323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e34af-c3ff-4315-83f7-3812157362ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:08.358692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:08.359646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:08.360146 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:08.360590 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] PATH_INFO: `/v3/projects/a4709278e71a4e1fadde5e4d227941fd` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:08.362227 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4709278e71a4e1fadde5e4d227941fd)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:08.366132 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-063e34af-c3ff-4315-83f7-3812157362ca None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:08.368492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-063e34af-c3ff-4315-83f7-3812157362ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:08.375340 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:08.394743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1248/2483] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:08.397404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e34af-c3ff-4315-83f7-3812157362ca None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:08.407917 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:08.421060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e34af-c3ff-4315-83f7-3812157362ca None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:08.447916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d88bb-c0a4-414f-abd4-cbb842ddadd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:08.450825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d88bb-c0a4-414f-abd4-cbb842ddadd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:08.451117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d88bb-c0a4-414f-abd4-cbb842ddadd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:08.451804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d88bb-c0a4-414f-abd4-cbb842ddadd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:08.454831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d88bb-c0a4-414f-abd4-cbb842ddadd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:08.455113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d88bb-c0a4-414f-abd4-cbb842ddadd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:08.460775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:08.462208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d88bb-c0a4-414f-abd4-cbb842ddadd6 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:08.505103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1249/2484] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:19:08.506471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d88bb-c0a4-414f-abd4-cbb842ddadd6 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:08.543751 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4709278e71a4e1fadde5e4d227941fd', 'request_id': 'req-266d8a2c-8fbf-4763-894e-a7cec8259036'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:08.547682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d88bb-c0a4-414f-abd4-cbb842ddadd6 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:08.553379 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4709278e71a4e1fadde5e4d227941fd', 'request_id': 'req-266d8a2c-8fbf-4763-894e-a7cec8259036'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:08.567301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4709278e71a4e1fadde5e4d227941fd', 'request_id': 'req-266d8a2c-8fbf-4763-894e-a7cec8259036'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:08.613001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] The token cache is being invalidate because project a4709278e71a4e1fadde5e4d227941fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:08.613313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:08.618951 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:08.623655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1236/2485] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:07 2022] DELETE /identity/v3/projects/a4709278e71a4e1fadde5e4d227941fd => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:08.763219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1328c59-1c1e-428e-bc29-280b54f635cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:08.764268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1328c59-1c1e-428e-bc29-280b54f635cf admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:08.764268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1328c59-1c1e-428e-bc29-280b54f635cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:08.764602 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1328c59-1c1e-428e-bc29-280b54f635cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:08.765113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1328c59-1c1e-428e-bc29-280b54f635cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:08.777148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1328c59-1c1e-428e-bc29-280b54f635cf admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:08.878260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1250/2486] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:08 2022] POST /identity/v3/projects => generated 385 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:08.881205 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:08.884994 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:08.931190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d8a2c-8fbf-4763-894e-a7cec8259036 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:09.045400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eae87-ac06-4265-9f1a-871dbbb64a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:09.046274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eae87-ac06-4265-9f1a-871dbbb64a4a admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:09.046577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eae87-ac06-4265-9f1a-871dbbb64a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:09.047000 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eae87-ac06-4265-9f1a-871dbbb64a4a admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:09.047631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7eae87-ac06-4265-9f1a-871dbbb64a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:09.050114 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7eae87-ac06-4265-9f1a-871dbbb64a4a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:09.090729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1237/2487] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:08 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:09.093280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1328c59-1c1e-428e-bc29-280b54f635cf admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:09.109993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1328c59-1c1e-428e-bc29-280b54f635cf admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:09.116047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efd219-e2bf-41a0-ac6c-a99b128d89d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:09.116512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efd219-e2bf-41a0-ac6c-a99b128d89d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:09.116892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efd219-e2bf-41a0-ac6c-a99b128d89d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:09.117261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efd219-e2bf-41a0-ac6c-a99b128d89d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:09.118029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efd219-e2bf-41a0-ac6c-a99b128d89d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:09.119950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efd219-e2bf-41a0-ac6c-a99b128d89d8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:09.140856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1251/2488] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:09.146138 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eae87-ac06-4265-9f1a-871dbbb64a4a admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:09.167625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eae87-ac06-4265-9f1a-871dbbb64a4a admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:09.178046 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e5987-ab49-4d50-9919-8a02195c893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:09.179200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e5987-ab49-4d50-9919-8a02195c893d admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:09.180564 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e5987-ab49-4d50-9919-8a02195c893d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:09.180564 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e5987-ab49-4d50-9919-8a02195c893d admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:09.180879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e5987-ab49-4d50-9919-8a02195c893d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:09.183830 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e5987-ab49-4d50-9919-8a02195c893d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:09.205611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1238/2489] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:09.210066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efd219-e2bf-41a0-ac6c-a99b128d89d8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:09.240808 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efd219-e2bf-41a0-ac6c-a99b128d89d8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:09.291678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48319df-e302-4c17-914e-cf735d0e994a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:09.294020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48319df-e302-4c17-914e-cf735d0e994a admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:09.294312 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48319df-e302-4c17-914e-cf735d0e994a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:09.294592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48319df-e302-4c17-914e-cf735d0e994a admin admin] PATH_INFO: `/v3/projects/cd95b512e35e488fb7ee9d8e9631609c/users/a2964155b2df4868809d6a36773d4764/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:09.351965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48319df-e302-4c17-914e-cf735d0e994a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd95b512e35e488fb7ee9d8e9631609c, user_id=a2964155b2df4868809d6a36773d4764, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:09.368773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48319df-e302-4c17-914e-cf735d0e994a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:09.411028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1252/2490] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:09 2022] PUT /identity/v3/projects/cd95b512e35e488fb7ee9d8e9631609c/users/a2964155b2df4868809d6a36773d4764/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:09.423420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4d18f-435e-48dd-b4b6-61a1cd25c73d None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:09.423681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4d18f-435e-48dd-b4b6-61a1cd25c73d None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:09.424171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4d18f-435e-48dd-b4b6-61a1cd25c73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:09.483842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-ccf4d18f-435e-48dd-b4b6-61a1cd25c73d None None] MFA Rules not processed for user `a2964155b2df4868809d6a36773d4764`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:09.485198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ccf4d18f-435e-48dd-b4b6-61a1cd25c73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:09.673035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1239/2491] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:09.682188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48319df-e302-4c17-914e-cf735d0e994a admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:09.701297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b48319df-e302-4c17-914e-cf735d0e994a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:09.862573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48319df-e302-4c17-914e-cf735d0e994a admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:09.866920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eb64c-6a99-483e-91ab-9fca3fea0ef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:09.867755 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eb64c-6a99-483e-91ab-9fca3fea0ef4 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:09.868259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eb64c-6a99-483e-91ab-9fca3fea0ef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:09.868790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eb64c-6a99-483e-91ab-9fca3fea0ef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:09.893383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7eb64c-6a99-483e-91ab-9fca3fea0ef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:09.898704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7eb64c-6a99-483e-91ab-9fca3fea0ef4 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:09.975825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1253/2492] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:13.375154 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5ea8e-52ff-484f-abd4-2bccdda45261 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:13.375711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5ea8e-52ff-484f-abd4-2bccdda45261 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:13.375711 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5ea8e-52ff-484f-abd4-2bccdda45261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:13.405210 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-a4e5ea8e-52ff-484f-abd4-2bccdda45261 None None] MFA Rules not processed for user `b5398b12577a41cb9891845bc3671f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:13.406684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a4e5ea8e-52ff-484f-abd4-2bccdda45261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:13.410958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:13.494067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1240/2493] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:13.503168 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eb64c-6a99-483e-91ab-9fca3fea0ef4 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:13.505153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6f7eb64c-6a99-483e-91ab-9fca3fea0ef4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:13.591107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:13.659186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eb64c-6a99-483e-91ab-9fca3fea0ef4 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:13.671835 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe53c7f-b6c1-4d9f-a285-c03180973438 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:13.672796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe53c7f-b6c1-4d9f-a285-c03180973438 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:13.673360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe53c7f-b6c1-4d9f-a285-c03180973438 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:13.673360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe53c7f-b6c1-4d9f-a285-c03180973438 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:13.676352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ebe53c7f-b6c1-4d9f-a285-c03180973438 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:13.808419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe53c7f-b6c1-4d9f-a285-c03180973438 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:13.810260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe53c7f-b6c1-4d9f-a285-c03180973438 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:13.820393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1254/2494] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:23.482674 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=27 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:23.815796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=37 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:26.597429 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5ea8e-52ff-484f-abd4-2bccdda45261 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:26.706456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5ea8e-52ff-484f-abd4-2bccdda45261 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:26.711699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:26.713150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:26.713437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:26.713709 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] PATH_INFO: `/v3/users/a2964155b2df4868809d6a36773d4764` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:26.714962 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2964155b2df4868809d6a36773d4764)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:26.717041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:26.771450 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2964155b2df4868809d6a36773d4764'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.782957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2964155b2df4868809d6a36773d4764'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.782957 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2964155b2df4868809d6a36773d4764'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.783332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2964155b2df4868809d6a36773d4764'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.798079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2964155b2df4868809d6a36773d4764'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.810490 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1241/2495] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:26 2022] DELETE /identity/v3/users/a2964155b2df4868809d6a36773d4764 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:26.815253 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfedf4-622f-44c0-94ad-8f2c5527da96 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:26.815794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfedf4-622f-44c0-94ad-8f2c5527da96 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:26.815794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfedf4-622f-44c0-94ad-8f2c5527da96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:26.837824 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-29dfedf4-622f-44c0-94ad-8f2c5527da96 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:26.839098 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-29dfedf4-622f-44c0-94ad-8f2c5527da96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:26.919447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1255/2496] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:26.929755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:26.932462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:26.997754 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cbf17-298f-44bf-a767-c96e6acfd8b3 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:27.002238 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a316e-3f9e-489a-a81c-c208f184fd72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:27.002878 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a316e-3f9e-489a-a81c-c208f184fd72 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:27.003157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a316e-3f9e-489a-a81c-c208f184fd72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:27.003430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a316e-3f9e-489a-a81c-c208f184fd72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:27.014742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a316e-3f9e-489a-a81c-c208f184fd72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:27.016233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a316e-3f9e-489a-a81c-c208f184fd72 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:27.028428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1242/2497] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:27.381772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfedf4-622f-44c0-94ad-8f2c5527da96 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:27.383377 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-29dfedf4-622f-44c0-94ad-8f2c5527da96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:27.405158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfedf4-622f-44c0-94ad-8f2c5527da96 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:27.421548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:27.423018 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:27.423549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:27.423950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1 admin admin] PATH_INFO: `/v3/projects/cd95b512e35e488fb7ee9d8e9631609c` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:27.426752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd95b512e35e488fb7ee9d8e9631609c)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:27.438082 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:27.514122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd95b512e35e488fb7ee9d8e9631609c', 'request_id': 'req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:27.526861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd95b512e35e488fb7ee9d8e9631609c', 'request_id': 'req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:27.538153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd95b512e35e488fb7ee9d8e9631609c', 'request_id': 'req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:27.559536 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1 admin admin] The token cache is being invalidate because project cd95b512e35e488fb7ee9d8e9631609c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:27.559930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:27.561675 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b0136fd2-e20c-4255-ad4d-88e306d5d2c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:27.562945 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1256/2498] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:27 2022] DELETE /identity/v3/projects/cd95b512e35e488fb7ee9d8e9631609c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:27.568805 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bdadb-84cd-4f7c-985b-5a3f502f01f2 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:27.569389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bdadb-84cd-4f7c-985b-5a3f502f01f2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:27.569822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bdadb-84cd-4f7c-985b-5a3f502f01f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:27.596620 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-e71bdadb-84cd-4f7c-985b-5a3f502f01f2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:27.597989 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e71bdadb-84cd-4f7c-985b-5a3f502f01f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:27.628535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:27.697255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1243/2499] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:27.711462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae9cfd-fbea-463e-a868-ab974a3be927 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:27.712926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae9cfd-fbea-463e-a868-ab974a3be927 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:27.713440 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae9cfd-fbea-463e-a868-ab974a3be927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:27.738311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6fae9cfd-fbea-463e-a868-ab974a3be927 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:27.739588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6fae9cfd-fbea-463e-a868-ab974a3be927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:27.749666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1257/2500] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:27.755719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bdadb-84cd-4f7c-985b-5a3f502f01f2 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:27.771101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bdadb-84cd-4f7c-985b-5a3f502f01f2 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:27.784587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a052c-2dd7-4213-9e50-b6a5655353cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:27.785246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a052c-2dd7-4213-9e50-b6a5655353cb admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:27.785523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a052c-2dd7-4213-9e50-b6a5655353cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:27.785788 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a052c-2dd7-4213-9e50-b6a5655353cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:27.786952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a052c-2dd7-4213-9e50-b6a5655353cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:27.786952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a052c-2dd7-4213-9e50-b6a5655353cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:27.788562 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a052c-2dd7-4213-9e50-b6a5655353cb admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:27.802962 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1244/2501] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:19:27.808357 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae9cfd-fbea-463e-a868-ab974a3be927 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:27.819002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae9cfd-fbea-463e-a868-ab974a3be927 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:27.824777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92373162-de84-4e2b-89e1-7533a0da5bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:27.825387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92373162-de84-4e2b-89e1-7533a0da5bed admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:27.825664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92373162-de84-4e2b-89e1-7533a0da5bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:27.825930 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92373162-de84-4e2b-89e1-7533a0da5bed admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:27.826654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92373162-de84-4e2b-89e1-7533a0da5bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:27.828898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92373162-de84-4e2b-89e1-7533a0da5bed admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:27.863015 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1258/2502] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:27 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:27.867241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a052c-2dd7-4213-9e50-b6a5655353cb admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:27.879924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a052c-2dd7-4213-9e50-b6a5655353cb admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:27.949308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25847bc6-8340-4271-a048-1f20283fc00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:27.949940 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25847bc6-8340-4271-a048-1f20283fc00a admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:27.950211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25847bc6-8340-4271-a048-1f20283fc00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:27.950470 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25847bc6-8340-4271-a048-1f20283fc00a admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:27.951136 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25847bc6-8340-4271-a048-1f20283fc00a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:27.953098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25847bc6-8340-4271-a048-1f20283fc00a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:27.977405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1245/2503] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:27 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:19:27.982625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92373162-de84-4e2b-89e1-7533a0da5bed admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:27.993665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92373162-de84-4e2b-89e1-7533a0da5bed admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:27.998500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf13d3-127b-4f7e-8244-36149edc1524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:27.999073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf13d3-127b-4f7e-8244-36149edc1524 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:27.999073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf13d3-127b-4f7e-8244-36149edc1524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:27.999595 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf13d3-127b-4f7e-8244-36149edc1524 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.000296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf13d3-127b-4f7e-8244-36149edc1524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.001922 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf13d3-127b-4f7e-8244-36149edc1524 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.014223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1259/2504] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:28.019011 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25847bc6-8340-4271-a048-1f20283fc00a admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.037910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25847bc6-8340-4271-a048-1f20283fc00a admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.042948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd445f-5ffa-4287-9e73-5f228506a4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.043564 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd445f-5ffa-4287-9e73-5f228506a4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.043873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd445f-5ffa-4287-9e73-5f228506a4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.044175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd445f-5ffa-4287-9e73-5f228506a4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.044843 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd445f-5ffa-4287-9e73-5f228506a4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.046525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd445f-5ffa-4287-9e73-5f228506a4b8 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.059430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1246/2505] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:28.063943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf13d3-127b-4f7e-8244-36149edc1524 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.076436 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf13d3-127b-4f7e-8244-36149edc1524 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.081154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271769d-7a46-4112-a119-fd750459be33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.081838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271769d-7a46-4112-a119-fd750459be33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.082143 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271769d-7a46-4112-a119-fd750459be33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.082565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271769d-7a46-4112-a119-fd750459be33 admin admin] PATH_INFO: `/v3/projects/e472da4bba6c4458aba0b88d638c24f4/users/f66b9b8b2b8044e0a0fc7a33ba1122de/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.100221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271769d-7a46-4112-a119-fd750459be33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e472da4bba6c4458aba0b88d638c24f4, user_id=f66b9b8b2b8044e0a0fc7a33ba1122de, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.103477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271769d-7a46-4112-a119-fd750459be33 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.121051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1260/2506] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:28 2022] PUT /identity/v3/projects/e472da4bba6c4458aba0b88d638c24f4/users/f66b9b8b2b8044e0a0fc7a33ba1122de/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:28.133641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b579ce-db2d-4bfb-880b-28405cf2608f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.135322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b579ce-db2d-4bfb-880b-28405cf2608f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.135322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b579ce-db2d-4bfb-880b-28405cf2608f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.187201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-f7b579ce-db2d-4bfb-880b-28405cf2608f None None] MFA Rules not processed for user `f66b9b8b2b8044e0a0fc7a33ba1122de`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:28.189073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f7b579ce-db2d-4bfb-880b-28405cf2608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:28.281995 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1247/2507] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:28.286725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271769d-7a46-4112-a119-fd750459be33 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.312514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:28.365395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271769d-7a46-4112-a119-fd750459be33 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.370825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c13cd-2abd-4515-b99c-6250ef51ba18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.371450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c13cd-2abd-4515-b99c-6250ef51ba18 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.371722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c13cd-2abd-4515-b99c-6250ef51ba18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.372014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c13cd-2abd-4515-b99c-6250ef51ba18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.372683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c13cd-2abd-4515-b99c-6250ef51ba18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.374608 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c13cd-2abd-4515-b99c-6250ef51ba18 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.411409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1261/2508] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:28 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:28.415854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b579ce-db2d-4bfb-880b-28405cf2608f None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.432958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b579ce-db2d-4bfb-880b-28405cf2608f None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.449302 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:28.509393 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889524e-11e9-4cde-8dee-d1bd0372ed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.510128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889524e-11e9-4cde-8dee-d1bd0372ed2f admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.510357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889524e-11e9-4cde-8dee-d1bd0372ed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.510629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889524e-11e9-4cde-8dee-d1bd0372ed2f admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.511789 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889524e-11e9-4cde-8dee-d1bd0372ed2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.513764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889524e-11e9-4cde-8dee-d1bd0372ed2f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.545124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1248/2509] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:28 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:19:28.550583 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c13cd-2abd-4515-b99c-6250ef51ba18 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.566167 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c13cd-2abd-4515-b99c-6250ef51ba18 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.572905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabbfed-afdf-4404-9563-db5a633e6732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.573779 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabbfed-afdf-4404-9563-db5a633e6732 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.574174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabbfed-afdf-4404-9563-db5a633e6732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.574658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabbfed-afdf-4404-9563-db5a633e6732 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.575797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aabbfed-afdf-4404-9563-db5a633e6732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.577785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aabbfed-afdf-4404-9563-db5a633e6732 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.591761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1262/2510] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:28.596619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889524e-11e9-4cde-8dee-d1bd0372ed2f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.607873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889524e-11e9-4cde-8dee-d1bd0372ed2f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.613801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d227bc1-70c6-4e27-a5b2-7d045a9a7dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.614535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d227bc1-70c6-4e27-a5b2-7d045a9a7dfa admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.614818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d227bc1-70c6-4e27-a5b2-7d045a9a7dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.615092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d227bc1-70c6-4e27-a5b2-7d045a9a7dfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.615859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d227bc1-70c6-4e27-a5b2-7d045a9a7dfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.617524 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d227bc1-70c6-4e27-a5b2-7d045a9a7dfa admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.636605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1249/2511] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:28.642670 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabbfed-afdf-4404-9563-db5a633e6732 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.660232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabbfed-afdf-4404-9563-db5a633e6732 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.668112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea000d6-7301-4479-9d48-8cde8581088f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.669183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea000d6-7301-4479-9d48-8cde8581088f admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.669634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea000d6-7301-4479-9d48-8cde8581088f admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.670072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea000d6-7301-4479-9d48-8cde8581088f admin admin] PATH_INFO: `/v3/projects/91039ad2b1b74bfb8df30443ec02c76e/users/c2414d6f2128415b849629825ed1d9ca/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.704279 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea000d6-7301-4479-9d48-8cde8581088f admin admin] RBAC: Authorizing `identity:create_grant(project_id=91039ad2b1b74bfb8df30443ec02c76e, user_id=c2414d6f2128415b849629825ed1d9ca, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.707250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea000d6-7301-4479-9d48-8cde8581088f admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.736567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1263/2512] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:28 2022] PUT /identity/v3/projects/91039ad2b1b74bfb8df30443ec02c76e/users/c2414d6f2128415b849629825ed1d9ca/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:28.745176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e142f-e96a-491e-8d2f-a832b71f153e None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.745624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e142f-e96a-491e-8d2f-a832b71f153e None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.746002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e142f-e96a-491e-8d2f-a832b71f153e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.780402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-778e142f-e96a-491e-8d2f-a832b71f153e None None] MFA Rules not processed for user `c2414d6f2128415b849629825ed1d9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:28.781661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-778e142f-e96a-491e-8d2f-a832b71f153e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:28.865200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1250/2513] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:28.870317 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea000d6-7301-4479-9d48-8cde8581088f admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.951621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea000d6-7301-4479-9d48-8cde8581088f admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.957400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce7af9-bfb5-4947-91bd-a35b39d7537b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.958116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce7af9-bfb5-4947-91bd-a35b39d7537b admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.958468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce7af9-bfb5-4947-91bd-a35b39d7537b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.958801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce7af9-bfb5-4947-91bd-a35b39d7537b admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.959490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce7af9-bfb5-4947-91bd-a35b39d7537b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.962656 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce7af9-bfb5-4947-91bd-a35b39d7537b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.994903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1264/2514] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:28 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:28.999870 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e142f-e96a-491e-8d2f-a832b71f153e None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.017297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e142f-e96a-491e-8d2f-a832b71f153e None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.088694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692fe90-b87b-4bdd-bb62-f3a76101269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.089317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692fe90-b87b-4bdd-bb62-f3a76101269f admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.089592 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692fe90-b87b-4bdd-bb62-f3a76101269f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.089866 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692fe90-b87b-4bdd-bb62-f3a76101269f admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.090506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692fe90-b87b-4bdd-bb62-f3a76101269f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.092496 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692fe90-b87b-4bdd-bb62-f3a76101269f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.120217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1251/2515] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:28 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:29.121588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce7af9-bfb5-4947-91bd-a35b39d7537b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.133173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce7af9-bfb5-4947-91bd-a35b39d7537b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.138530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24002faf-d4d8-4c3d-b852-acc7955e023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.139183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24002faf-d4d8-4c3d-b852-acc7955e023f admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.139495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24002faf-d4d8-4c3d-b852-acc7955e023f admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.139810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24002faf-d4d8-4c3d-b852-acc7955e023f admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.140537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24002faf-d4d8-4c3d-b852-acc7955e023f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.142283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24002faf-d4d8-4c3d-b852-acc7955e023f admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.156139 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1265/2516] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:29.160627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692fe90-b87b-4bdd-bb62-f3a76101269f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.171798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692fe90-b87b-4bdd-bb62-f3a76101269f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.176905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6698ea2-e890-4ef0-8396-02396881491b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.177580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6698ea2-e890-4ef0-8396-02396881491b admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.177854 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6698ea2-e890-4ef0-8396-02396881491b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.178130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6698ea2-e890-4ef0-8396-02396881491b admin admin] PATH_INFO: `/v3/projects/efbf5bff88974dccba496a1a884b0683/users/25ce211065514225a03f476cb31190ff/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.192607 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:29.196139 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6698ea2-e890-4ef0-8396-02396881491b admin admin] RBAC: Authorizing `identity:create_grant(project_id=efbf5bff88974dccba496a1a884b0683, user_id=25ce211065514225a03f476cb31190ff, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.200016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6698ea2-e890-4ef0-8396-02396881491b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.214911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1252/2517] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:29 2022] PUT /identity/v3/projects/efbf5bff88974dccba496a1a884b0683/users/25ce211065514225a03f476cb31190ff/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:29.219010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24002faf-d4d8-4c3d-b852-acc7955e023f admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.229235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24002faf-d4d8-4c3d-b852-acc7955e023f admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.277322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:29.303442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc2643-0262-44e6-8e8f-e9ac524b312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.304204 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc2643-0262-44e6-8e8f-e9ac524b312a admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.304687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc2643-0262-44e6-8e8f-e9ac524b312a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.304972 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc2643-0262-44e6-8e8f-e9ac524b312a admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.306182 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc2643-0262-44e6-8e8f-e9ac524b312a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.308396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc2643-0262-44e6-8e8f-e9ac524b312a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.319552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1266/2518] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:29.325303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6698ea2-e890-4ef0-8396-02396881491b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.336861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6698ea2-e890-4ef0-8396-02396881491b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.341544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd538a0-a90f-42ff-a59d-fbc8949b5628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.342210 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd538a0-a90f-42ff-a59d-fbc8949b5628 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.342487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd538a0-a90f-42ff-a59d-fbc8949b5628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.342761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd538a0-a90f-42ff-a59d-fbc8949b5628 admin admin] PATH_INFO: `/v3/projects/efbf5bff88974dccba496a1a884b0683/users/25ce211065514225a03f476cb31190ff/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.344942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd538a0-a90f-42ff-a59d-fbc8949b5628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efbf5bff88974dccba496a1a884b0683, user_id=25ce211065514225a03f476cb31190ff, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.347642 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd538a0-a90f-42ff-a59d-fbc8949b5628 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.359546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1253/2519] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:29 2022] PUT /identity/v3/projects/efbf5bff88974dccba496a1a884b0683/users/25ce211065514225a03f476cb31190ff/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:29.372111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3c907-8c14-445b-9afa-ea7f894ff0bf None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.372392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3c907-8c14-445b-9afa-ea7f894ff0bf None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.372522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3c907-8c14-445b-9afa-ea7f894ff0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.404125 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-27b3c907-8c14-445b-9afa-ea7f894ff0bf None None] MFA Rules not processed for user `25ce211065514225a03f476cb31190ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:29.405418 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-27b3c907-8c14-445b-9afa-ea7f894ff0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:29.483290 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1267/2520] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:29.496518 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd538a0-a90f-42ff-a59d-fbc8949b5628 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.498423 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-afd538a0-a90f-42ff-a59d-fbc8949b5628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:29.560171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd538a0-a90f-42ff-a59d-fbc8949b5628 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.564130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208a2e2-24f6-46fe-9a63-119e7adfc8f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.564739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208a2e2-24f6-46fe-9a63-119e7adfc8f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.565015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208a2e2-24f6-46fe-9a63-119e7adfc8f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.565284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208a2e2-24f6-46fe-9a63-119e7adfc8f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.623727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208a2e2-24f6-46fe-9a63-119e7adfc8f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.625093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208a2e2-24f6-46fe-9a63-119e7adfc8f5 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.635609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1254/2521] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:31.306286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3c907-8c14-445b-9afa-ea7f894ff0bf None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:31.318421 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3c907-8c14-445b-9afa-ea7f894ff0bf None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:31.325079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5f8a9-070b-4e89-be7c-eb2d893f6214 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:31.325739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5f8a9-070b-4e89-be7c-eb2d893f6214 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:31.326116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5f8a9-070b-4e89-be7c-eb2d893f6214 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:31.326525 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5f8a9-070b-4e89-be7c-eb2d893f6214 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:31.391031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5f8a9-070b-4e89-be7c-eb2d893f6214 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:31.392647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5f8a9-070b-4e89-be7c-eb2d893f6214 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:31.403950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1268/2522] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:32.120927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208a2e2-24f6-46fe-9a63-119e7adfc8f5 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:32.122426 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e208a2e2-24f6-46fe-9a63-119e7adfc8f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:32.134071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208a2e2-24f6-46fe-9a63-119e7adfc8f5 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:32.137735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e2c2f-c169-41a9-9729-8486620a9e42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:32.138338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e2c2f-c169-41a9-9729-8486620a9e42 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:32.138616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e2c2f-c169-41a9-9729-8486620a9e42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:32.138882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e2c2f-c169-41a9-9729-8486620a9e42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:32.147821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e2c2f-c169-41a9-9729-8486620a9e42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:32.158034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e2c2f-c169-41a9-9729-8486620a9e42 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:32.162355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1255/2523] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:36.344761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5f8a9-070b-4e89-be7c-eb2d893f6214 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:36.446328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5f8a9-070b-4e89-be7c-eb2d893f6214 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:36.451028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39d779-28fe-4b12-aad6-783d723a99cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:36.451687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39d779-28fe-4b12-aad6-783d723a99cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:36.451962 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39d779-28fe-4b12-aad6-783d723a99cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:36.452275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39d779-28fe-4b12-aad6-783d723a99cc admin admin] PATH_INFO: `/v3/users/f66b9b8b2b8044e0a0fc7a33ba1122de` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:36.453454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39d779-28fe-4b12-aad6-783d723a99cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66b9b8b2b8044e0a0fc7a33ba1122de)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:36.455438 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39d779-28fe-4b12-aad6-783d723a99cc admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:36.461103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:36.532256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2b39d779-28fe-4b12-aad6-783d723a99cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66b9b8b2b8044e0a0fc7a33ba1122de'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:36.554502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e2c2f-c169-41a9-9729-8486620a9e42 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:36.558322 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-874e2c2f-c169-41a9-9729-8486620a9e42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:36.578586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2b39d779-28fe-4b12-aad6-783d723a99cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66b9b8b2b8044e0a0fc7a33ba1122de'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:36.602415 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2b39d779-28fe-4b12-aad6-783d723a99cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66b9b8b2b8044e0a0fc7a33ba1122de'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:36.619741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2b39d779-28fe-4b12-aad6-783d723a99cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66b9b8b2b8044e0a0fc7a33ba1122de'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:36.620900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2b39d779-28fe-4b12-aad6-783d723a99cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66b9b8b2b8044e0a0fc7a33ba1122de'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:36.637150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1269/2524] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:36 2022] DELETE /identity/v3/users/f66b9b8b2b8044e0a0fc7a33ba1122de => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:36.644052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d5024-0fef-42ca-b718-911347fe0d61 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:36.644578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d5024-0fef-42ca-b718-911347fe0d61 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:36.645080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d5024-0fef-42ca-b718-911347fe0d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:36.676837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-045d5024-0fef-42ca-b718-911347fe0d61 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:36.681270 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-045d5024-0fef-42ca-b718-911347fe0d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:36.705850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e2c2f-c169-41a9-9729-8486620a9e42 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:36.754823 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:36.803248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1270/2525] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:36.814099 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d5024-0fef-42ca-b718-911347fe0d61 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:36.816062 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-045d5024-0fef-42ca-b718-911347fe0d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:36.840823 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17999355-1897-4c29-bab2-0489e834221e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08f34295f19b4d6ea405f1e8c8190db5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:36.841684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17999355-1897-4c29-bab2-0489e834221e service placement] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:36.842124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17999355-1897-4c29-bab2-0489e834221e service placement] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:36.842555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17999355-1897-4c29-bab2-0489e834221e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:36.845334 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-17999355-1897-4c29-bab2-0489e834221e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:36.864869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17999355-1897-4c29-bab2-0489e834221e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:36.868260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17999355-1897-4c29-bab2-0489e834221e service placement] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:36.888728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1256/2526] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:36.932068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d5024-0fef-42ca-b718-911347fe0d61 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:36.937981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c18cf-5f31-4a07-bdad-ba775e89a930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:36.938958 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c18cf-5f31-4a07-bdad-ba775e89a930 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:36.939468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c18cf-5f31-4a07-bdad-ba775e89a930 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:36.940234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c18cf-5f31-4a07-bdad-ba775e89a930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:36.954964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c18cf-5f31-4a07-bdad-ba775e89a930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:36.957212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c18cf-5f31-4a07-bdad-ba775e89a930 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:36.970117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1271/2527] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:19:37.317308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17999355-1897-4c29-bab2-0489e834221e service placement] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:37.318927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-17999355-1897-4c29-bab2-0489e834221e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:37.350490 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17999355-1897-4c29-bab2-0489e834221e service placement] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:37.357502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:37.358509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:37.358890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:37.359272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] PATH_INFO: `/v3/projects/e472da4bba6c4458aba0b88d638c24f4` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:37.360556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e472da4bba6c4458aba0b88d638c24f4)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:37.362831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:37.442833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e472da4bba6c4458aba0b88d638c24f4', 'request_id': 'req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.451065 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e472da4bba6c4458aba0b88d638c24f4', 'request_id': 'req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.463078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e472da4bba6c4458aba0b88d638c24f4', 'request_id': 'req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.484539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] The token cache is being invalidate because project e472da4bba6c4458aba0b88d638c24f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:37.485059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.486437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.488627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1257/2528] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:37 2022] DELETE /identity/v3/projects/e472da4bba6c4458aba0b88d638c24f4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:37.492871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c18cf-5f31-4a07-bdad-ba775e89a930 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:37.502734 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c18cf-5f31-4a07-bdad-ba775e89a930 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:37.535319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:37.572486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:37.573126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:37.573398 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:37.573664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] PATH_INFO: `/v3/users/c2414d6f2128415b849629825ed1d9ca` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:37.574805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2414d6f2128415b849629825ed1d9ca)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:37.576772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:37.639583 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2414d6f2128415b849629825ed1d9ca'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.654909 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2414d6f2128415b849629825ed1d9ca'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.662550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2414d6f2128415b849629825ed1d9ca'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.673796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2414d6f2128415b849629825ed1d9ca'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.674516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2414d6f2128415b849629825ed1d9ca'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.681640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1272/2529] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:37 2022] DELETE /identity/v3/users/c2414d6f2128415b849629825ed1d9ca => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:38.036975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:38.038451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:38.067865 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a9cfa-1f6a-4af7-bf87-aaf7ff094d56 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:38.155037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:38.155622 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.155900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.156280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] PATH_INFO: `/v3/projects/91039ad2b1b74bfb8df30443ec02c76e` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.157368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] RBAC: Authorizing `identity:delete_project(project_id=91039ad2b1b74bfb8df30443ec02c76e)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:38.159321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:38.210785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:38.229877 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91039ad2b1b74bfb8df30443ec02c76e', 'request_id': 'req-08c969e9-0d09-42f5-9cf8-988548677fae'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:38.240759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91039ad2b1b74bfb8df30443ec02c76e', 'request_id': 'req-08c969e9-0d09-42f5-9cf8-988548677fae'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:38.253263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91039ad2b1b74bfb8df30443ec02c76e', 'request_id': 'req-08c969e9-0d09-42f5-9cf8-988548677fae'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:38.286529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] The token cache is being invalidate because project 91039ad2b1b74bfb8df30443ec02c76e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:38.286848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:38.287636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:38.290281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1258/2530] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:38 2022] DELETE /identity/v3/projects/91039ad2b1b74bfb8df30443ec02c76e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:38.294767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:38.296337 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:38.311761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9dcbe-e819-45fd-9c62-0c2ed8f038c9 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:38.399766 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:38.401451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.401959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.402421 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] PATH_INFO: `/v3/users/25ce211065514225a03f476cb31190ff` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.403987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25ce211065514225a03f476cb31190ff)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:38.407625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:38.485534 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25ce211065514225a03f476cb31190ff'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:38.488084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:38.505131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25ce211065514225a03f476cb31190ff'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:38.522150 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25ce211065514225a03f476cb31190ff'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:38.531409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25ce211065514225a03f476cb31190ff'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:38.532239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25ce211065514225a03f476cb31190ff'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:38.542053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1273/2531] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:38 2022] DELETE /identity/v3/users/25ce211065514225a03f476cb31190ff => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:39.111896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:39.113510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:39.135317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c969e9-0d09-42f5-9cf8-988548677fae admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:39.244625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52939c9-57da-450c-a061-78604375ca3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:39.245639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52939c9-57da-450c-a061-78604375ca3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:39.246142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52939c9-57da-450c-a061-78604375ca3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:39.246674 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52939c9-57da-450c-a061-78604375ca3e admin admin] PATH_INFO: `/v3/projects/efbf5bff88974dccba496a1a884b0683` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:39.248875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52939c9-57da-450c-a061-78604375ca3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=efbf5bff88974dccba496a1a884b0683)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:39.252861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52939c9-57da-450c-a061-78604375ca3e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:39.328546 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b52939c9-57da-450c-a061-78604375ca3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efbf5bff88974dccba496a1a884b0683', 'request_id': 'req-b52939c9-57da-450c-a061-78604375ca3e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:39.340610 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b52939c9-57da-450c-a061-78604375ca3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efbf5bff88974dccba496a1a884b0683', 'request_id': 'req-b52939c9-57da-450c-a061-78604375ca3e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:39.349527 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b52939c9-57da-450c-a061-78604375ca3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efbf5bff88974dccba496a1a884b0683', 'request_id': 'req-b52939c9-57da-450c-a061-78604375ca3e'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:39.376850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b52939c9-57da-450c-a061-78604375ca3e admin admin] The token cache is being invalidate because project efbf5bff88974dccba496a1a884b0683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:39.377320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b52939c9-57da-450c-a061-78604375ca3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:39.378181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b52939c9-57da-450c-a061-78604375ca3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:39.380446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1259/2532] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:39 2022] DELETE /identity/v3/projects/efbf5bff88974dccba496a1a884b0683 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:44.003423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:44.005026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:44.026242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd825f75-0cc7-4e9b-8378-b51f7d5d2105 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:44.105063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:44.105630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:44.105914 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:44.106360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] PATH_INFO: `/v3/users/c654d71744924651a9928fb428d0c93c` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:44.107711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c654d71744924651a9928fb428d0c93c)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:44.109806 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:44.185154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c654d71744924651a9928fb428d0c93c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:44.201919 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c654d71744924651a9928fb428d0c93c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:44.215645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c654d71744924651a9928fb428d0c93c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:44.241567 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c654d71744924651a9928fb428d0c93c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:44.242437 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c654d71744924651a9928fb428d0c93c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:44.254271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1274/2533] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:44 2022] DELETE /identity/v3/users/c654d71744924651a9928fb428d0c93c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:44.260473 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229afe2-944b-4dce-be46-de09fe6f0be6 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:44.261051 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229afe2-944b-4dce-be46-de09fe6f0be6 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:44.261403 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229afe2-944b-4dce-be46-de09fe6f0be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:44.300850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-c229afe2-944b-4dce-be46-de09fe6f0be6 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:44.302514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c229afe2-944b-4dce-be46-de09fe6f0be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:44.383016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:44.409107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1260/2534] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:44.421031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:44.422689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:44.541110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:44.563797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef6f69-9f16-4e40-ab30-74d2f77381bd admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:44.570842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641abf6c-b348-4c14-a4a9-957bbcdd7ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:44.571231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641abf6c-b348-4c14-a4a9-957bbcdd7ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:44.571645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641abf6c-b348-4c14-a4a9-957bbcdd7ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:44.572243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641abf6c-b348-4c14-a4a9-957bbcdd7ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:44.589555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641abf6c-b348-4c14-a4a9-957bbcdd7ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:44.591647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641abf6c-b348-4c14-a4a9-957bbcdd7ec8 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:44.606271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1275/2535] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:45.026962 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229afe2-944b-4dce-be46-de09fe6f0be6 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:45.029078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c229afe2-944b-4dce-be46-de09fe6f0be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:45.050299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229afe2-944b-4dce-be46-de09fe6f0be6 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:45.060464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fbe24-db79-4b72-be86-1ddc179bc896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:45.061821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fbe24-db79-4b72-be86-1ddc179bc896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:45.062380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fbe24-db79-4b72-be86-1ddc179bc896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:45.062951 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fbe24-db79-4b72-be86-1ddc179bc896 admin admin] PATH_INFO: `/v3/projects/a4e3c9191c3542bbb022d1ffdf052a92` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.065203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309fbe24-db79-4b72-be86-1ddc179bc896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e3c9191c3542bbb022d1ffdf052a92)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:45.069088 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309fbe24-db79-4b72-be86-1ddc179bc896 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:45.131412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-309fbe24-db79-4b72-be86-1ddc179bc896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e3c9191c3542bbb022d1ffdf052a92', 'request_id': 'req-309fbe24-db79-4b72-be86-1ddc179bc896'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:45.138225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-309fbe24-db79-4b72-be86-1ddc179bc896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e3c9191c3542bbb022d1ffdf052a92', 'request_id': 'req-309fbe24-db79-4b72-be86-1ddc179bc896'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:45.145193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-309fbe24-db79-4b72-be86-1ddc179bc896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e3c9191c3542bbb022d1ffdf052a92', 'request_id': 'req-309fbe24-db79-4b72-be86-1ddc179bc896'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:45.161415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-309fbe24-db79-4b72-be86-1ddc179bc896 admin admin] The token cache is being invalidate because project a4e3c9191c3542bbb022d1ffdf052a92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:45.161701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-309fbe24-db79-4b72-be86-1ddc179bc896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:45.162372 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-309fbe24-db79-4b72-be86-1ddc179bc896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:45.164545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1261/2536] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:45 2022] DELETE /identity/v3/projects/a4e3c9191c3542bbb022d1ffdf052a92 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:45.169840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207db882-03df-4cec-a300-cf4fb9b2f35c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:45.170410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207db882-03df-4cec-a300-cf4fb9b2f35c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:45.170410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207db882-03df-4cec-a300-cf4fb9b2f35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.191020 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-207db882-03df-4cec-a300-cf4fb9b2f35c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:45.192264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-207db882-03df-4cec-a300-cf4fb9b2f35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:45.257008 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1276/2537] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:45.269788 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419c979-be10-450f-bc26-8df2c4b843d1 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:45.270335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419c979-be10-450f-bc26-8df2c4b843d1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:45.270335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419c979-be10-450f-bc26-8df2c4b843d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.294265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-9419c979-be10-450f-bc26-8df2c4b843d1 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:45.295485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9419c979-be10-450f-bc26-8df2c4b843d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:45.303463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1262/2538] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:45.307936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207db882-03df-4cec-a300-cf4fb9b2f35c None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:45.317928 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207db882-03df-4cec-a300-cf4fb9b2f35c None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:45.324384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036dc69-e001-44c5-bb46-bd330ff1f9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:45.325090 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036dc69-e001-44c5-bb46-bd330ff1f9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:45.325374 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036dc69-e001-44c5-bb46-bd330ff1f9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:45.325643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036dc69-e001-44c5-bb46-bd330ff1f9e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.326298 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036dc69-e001-44c5-bb46-bd330ff1f9e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:45.326603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036dc69-e001-44c5-bb46-bd330ff1f9e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:45.328231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036dc69-e001-44c5-bb46-bd330ff1f9e6 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:45.342500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1277/2539] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:19:45.346881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419c979-be10-450f-bc26-8df2c4b843d1 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:45.363249 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419c979-be10-450f-bc26-8df2c4b843d1 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:45.369628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff4f90-50a7-4ad6-ab0b-d7fb0be690d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:45.370246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff4f90-50a7-4ad6-ab0b-d7fb0be690d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:45.370525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff4f90-50a7-4ad6-ab0b-d7fb0be690d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:45.370798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff4f90-50a7-4ad6-ab0b-d7fb0be690d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.371729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff4f90-50a7-4ad6-ab0b-d7fb0be690d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:45.373690 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff4f90-50a7-4ad6-ab0b-d7fb0be690d8 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:45.402987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1263/2540] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:45 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:45.408469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036dc69-e001-44c5-bb46-bd330ff1f9e6 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:45.419727 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036dc69-e001-44c5-bb46-bd330ff1f9e6 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:45.480629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bd213-8058-4466-99cf-a6eaf52cd85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:45.481250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bd213-8058-4466-99cf-a6eaf52cd85f admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:45.481534 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bd213-8058-4466-99cf-a6eaf52cd85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:45.481809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bd213-8058-4466-99cf-a6eaf52cd85f admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.482464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bd213-8058-4466-99cf-a6eaf52cd85f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:45.484424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bd213-8058-4466-99cf-a6eaf52cd85f admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:45.508550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1278/2541] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:45 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:45.512684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff4f90-50a7-4ad6-ab0b-d7fb0be690d8 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:45.524336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff4f90-50a7-4ad6-ab0b-d7fb0be690d8 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:45.529153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a24509-b481-44d3-adcc-2a931e7c8de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:45.529680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a24509-b481-44d3-adcc-2a931e7c8de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:45.529956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a24509-b481-44d3-adcc-2a931e7c8de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:45.530231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a24509-b481-44d3-adcc-2a931e7c8de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.530872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a24509-b481-44d3-adcc-2a931e7c8de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:45.532656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a24509-b481-44d3-adcc-2a931e7c8de1 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:45.543604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1264/2542] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:45.549131 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bd213-8058-4466-99cf-a6eaf52cd85f admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:45.560412 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bd213-8058-4466-99cf-a6eaf52cd85f admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:45.565597 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e6542-b795-4bed-b49d-dba720256bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:45.566212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e6542-b795-4bed-b49d-dba720256bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:45.566485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e6542-b795-4bed-b49d-dba720256bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:45.566749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e6542-b795-4bed-b49d-dba720256bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.567398 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e6542-b795-4bed-b49d-dba720256bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:45.569060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e6542-b795-4bed-b49d-dba720256bc2 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:45.587200 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1279/2543] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:45.605002 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a24509-b481-44d3-adcc-2a931e7c8de1 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:45.622071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a24509-b481-44d3-adcc-2a931e7c8de1 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:45.627036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924388fa-7238-4e80-bdbf-bafcf6dc76a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:45.627741 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924388fa-7238-4e80-bdbf-bafcf6dc76a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:45.628038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924388fa-7238-4e80-bdbf-bafcf6dc76a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:45.628323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924388fa-7238-4e80-bdbf-bafcf6dc76a8 admin admin] PATH_INFO: `/v3/projects/91f4e4df4e2744bfb9fcd84431ed567e/users/a21183e645d74e4f9378c650e7916ba8/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.647245 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924388fa-7238-4e80-bdbf-bafcf6dc76a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f4e4df4e2744bfb9fcd84431ed567e, user_id=a21183e645d74e4f9378c650e7916ba8, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:45.650166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924388fa-7238-4e80-bdbf-bafcf6dc76a8 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:45.665557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1265/2544] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:45 2022] PUT /identity/v3/projects/91f4e4df4e2744bfb9fcd84431ed567e/users/a21183e645d74e4f9378c650e7916ba8/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:45.670709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e615a-7eb0-4202-b288-953ac23b33a2 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:45.671018 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e615a-7eb0-4202-b288-953ac23b33a2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:45.671295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e615a-7eb0-4202-b288-953ac23b33a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.697281 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ca7e615a-7eb0-4202-b288-953ac23b33a2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:45.698677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ca7e615a-7eb0-4202-b288-953ac23b33a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:45.755367 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:45.770114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1280/2545] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:45.780106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924388fa-7238-4e80-bdbf-bafcf6dc76a8 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:45.781884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-924388fa-7238-4e80-bdbf-bafcf6dc76a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:45.850850 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924388fa-7238-4e80-bdbf-bafcf6dc76a8 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:45.855207 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ec5d8-e694-43b0-8693-ed4a3b67111a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:45.855837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ec5d8-e694-43b0-8693-ed4a3b67111a service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:45.856144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ec5d8-e694-43b0-8693-ed4a3b67111a service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:45.856417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ec5d8-e694-43b0-8693-ed4a3b67111a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.867296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ec5d8-e694-43b0-8693-ed4a3b67111a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:45.868767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ec5d8-e694-43b0-8693-ed4a3b67111a service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:45.884401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1266/2546] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:19:47.771667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cef569-4563-4269-9457-41aeac056c9c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:47.772460 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cef569-4563-4269-9457-41aeac056c9c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:47.772460 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cef569-4563-4269-9457-41aeac056c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:47.805358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-77cef569-4563-4269-9457-41aeac056c9c None None] MFA Rules not processed for user `64b579837fb4434282224eb7b83e57ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:47.807648 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-77cef569-4563-4269-9457-41aeac056c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:47.903792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1281/2547] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:52.635425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e615fe-215b-441f-ad6c-56d9d5ceffe1 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:52.636420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e615fe-215b-441f-ad6c-56d9d5ceffe1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:52.636909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e615fe-215b-441f-ad6c-56d9d5ceffe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:52.674175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-84e615fe-215b-441f-ad6c-56d9d5ceffe1 None None] MFA Rules not processed for user `2839103f52324a1ab1ebd43fe230addc`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:52.676094 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-84e615fe-215b-441f-ad6c-56d9d5ceffe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:52.692366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:52.764628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1267/2548] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:53.003666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fe4b9-cfae-441e-93e1-8744562bccc6 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:53.003666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fe4b9-cfae-441e-93e1-8744562bccc6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:53.003666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fe4b9-cfae-441e-93e1-8744562bccc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:53.040718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e65fe4b9-cfae-441e-93e1-8744562bccc6 None None] MFA Rules not processed for user `a21183e645d74e4f9378c650e7916ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:53.042394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e65fe4b9-cfae-441e-93e1-8744562bccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:53.200113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1282/2549] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:19:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:53.203633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e615fe-215b-441f-ad6c-56d9d5ceffe1 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:53.227864 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e615fe-215b-441f-ad6c-56d9d5ceffe1 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:53.235602 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23e661-19a2-4afa-ba68-db5ce50b7249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:53.236679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23e661-19a2-4afa-ba68-db5ce50b7249 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:53.237226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23e661-19a2-4afa-ba68-db5ce50b7249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:53.237677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23e661-19a2-4afa-ba68-db5ce50b7249 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:53.238580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23e661-19a2-4afa-ba68-db5ce50b7249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:53.243130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23e661-19a2-4afa-ba68-db5ce50b7249 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:53.291217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1268/2550] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:53 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:19:53.293293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fe4b9-cfae-441e-93e1-8744562bccc6 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:53.311092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fe4b9-cfae-441e-93e1-8744562bccc6 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:53.388138 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fffea-098d-4af0-96d3-c4b3c238ffed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:53.389269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fffea-098d-4af0-96d3-c4b3c238ffed admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:53.389861 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fffea-098d-4af0-96d3-c4b3c238ffed admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:53.390283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fffea-098d-4af0-96d3-c4b3c238ffed admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:53.391102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fffea-098d-4af0-96d3-c4b3c238ffed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:53.393262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fffea-098d-4af0-96d3-c4b3c238ffed admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:53.420955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1283/2551] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:53 2022] POST /identity/v3/users => generated 298 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:53.425398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23e661-19a2-4afa-ba68-db5ce50b7249 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:53.443868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23e661-19a2-4afa-ba68-db5ce50b7249 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:53.455540 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32743def-ec63-4158-a0a6-c803f0849883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:53.456785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32743def-ec63-4158-a0a6-c803f0849883 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:53.456785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32743def-ec63-4158-a0a6-c803f0849883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:53.457096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32743def-ec63-4158-a0a6-c803f0849883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:53.457863 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32743def-ec63-4158-a0a6-c803f0849883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:53.463759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32743def-ec63-4158-a0a6-c803f0849883 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:53.476880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1269/2552] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:53.482325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fffea-098d-4af0-96d3-c4b3c238ffed admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:53.494610 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fffea-098d-4af0-96d3-c4b3c238ffed admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:53.499355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e6847-3b49-42d7-b3c7-30b937e44eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:53.500091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e6847-3b49-42d7-b3c7-30b937e44eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:53.500372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e6847-3b49-42d7-b3c7-30b937e44eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:53.500645 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e6847-3b49-42d7-b3c7-30b937e44eb9 admin admin] PATH_INFO: `/v3/projects/0ce5293d77734c08bdae2dd3a01ca285/users/0a9b2de11b724397ad569ee7f5372ff4/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:53.518776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e6847-3b49-42d7-b3c7-30b937e44eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce5293d77734c08bdae2dd3a01ca285, user_id=0a9b2de11b724397ad569ee7f5372ff4, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:53.522290 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e6847-3b49-42d7-b3c7-30b937e44eb9 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:53.525662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:53.550816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1284/2553] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:53 2022] PUT /identity/v3/projects/0ce5293d77734c08bdae2dd3a01ca285/users/0a9b2de11b724397ad569ee7f5372ff4/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:53.552783 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32743def-ec63-4158-a0a6-c803f0849883 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:53.576936 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32743def-ec63-4158-a0a6-c803f0849883 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:53.673284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6914313-e9c6-4f6b-b864-e2ee70405b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:53.674664 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6914313-e9c6-4f6b-b864-e2ee70405b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:53.675532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6914313-e9c6-4f6b-b864-e2ee70405b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:53.676022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6914313-e9c6-4f6b-b864-e2ee70405b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:53.677200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6914313-e9c6-4f6b-b864-e2ee70405b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:53.681536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6914313-e9c6-4f6b-b864-e2ee70405b4c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:53.709183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1270/2554] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:53.714498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e6847-3b49-42d7-b3c7-30b937e44eb9 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:53.730365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e6847-3b49-42d7-b3c7-30b937e44eb9 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:53.740212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab196b-a39c-4a08-be99-79fe385c50d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:53.741947 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab196b-a39c-4a08-be99-79fe385c50d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:53.742644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab196b-a39c-4a08-be99-79fe385c50d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:53.743165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab196b-a39c-4a08-be99-79fe385c50d3 admin admin] PATH_INFO: `/v3/projects/0ce5293d77734c08bdae2dd3a01ca285/users/0a9b2de11b724397ad569ee7f5372ff4/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:53.746919 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab196b-a39c-4a08-be99-79fe385c50d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce5293d77734c08bdae2dd3a01ca285, user_id=0a9b2de11b724397ad569ee7f5372ff4, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:53.752806 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab196b-a39c-4a08-be99-79fe385c50d3 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:53.774199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1285/2555] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:53 2022] PUT /identity/v3/projects/0ce5293d77734c08bdae2dd3a01ca285/users/0a9b2de11b724397ad569ee7f5372ff4/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:20:02.419373 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6914313-e9c6-4f6b-b864-e2ee70405b4c admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:20:02.422843 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b6914313-e9c6-4f6b-b864-e2ee70405b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:02.564486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6914313-e9c6-4f6b-b864-e2ee70405b4c admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:20:02.661575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:20:02.681045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa6458-a856-4b63-9a7e-2654f2d57027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:20:02.683699 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa6458-a856-4b63-9a7e-2654f2d57027 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:02.684189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa6458-a856-4b63-9a7e-2654f2d57027 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:02.684577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa6458-a856-4b63-9a7e-2654f2d57027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:02.694413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-33fa6458-a856-4b63-9a7e-2654f2d57027 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:02.722135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb22c0-f7d2-4e21-b985-76405a3715c6 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:02.722832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb22c0-f7d2-4e21-b985-76405a3715c6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:02.722832 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb22c0-f7d2-4e21-b985-76405a3715c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:02.765853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-8abb22c0-f7d2-4e21-b985-76405a3715c6 None None] MFA Rules not processed for user `0a9b2de11b724397ad569ee7f5372ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:20:02.769896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8abb22c0-f7d2-4e21-b985-76405a3715c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:02.864430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa6458-a856-4b63-9a7e-2654f2d57027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:20:02.866987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa6458-a856-4b63-9a7e-2654f2d57027 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:20:02.881936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1286/2556] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:20:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:20:02.885297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1271/2557] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:20:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:20:02.904221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb22c0-f7d2-4e21-b985-76405a3715c6 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:20:02.906401 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-8abb22c0-f7d2-4e21-b985-76405a3715c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:03.072046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb22c0-f7d2-4e21-b985-76405a3715c6 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:20:03.078635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27ef0f-1295-408e-81d5-23b4372c88ba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c9d444b3294ed18c3d488cf0a3eeb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:20:03.079785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27ef0f-1295-408e-81d5-23b4372c88ba service glance] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:03.080414 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27ef0f-1295-408e-81d5-23b4372c88ba service glance] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:03.080972 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27ef0f-1295-408e-81d5-23b4372c88ba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:03.232207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27ef0f-1295-408e-81d5-23b4372c88ba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:20:03.234055 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27ef0f-1295-408e-81d5-23b4372c88ba service glance] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:20:03.245638 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1287/2558] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:20:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:20:03.553323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc307a33-dd43-41a0-bc0a-7f24d2f47047 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:03.553969 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc307a33-dd43-41a0-bc0a-7f24d2f47047 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:03.554568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc307a33-dd43-41a0-bc0a-7f24d2f47047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:03.620759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-dc307a33-dd43-41a0-bc0a-7f24d2f47047 None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:20:03.622967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-dc307a33-dd43-41a0-bc0a-7f24d2f47047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:03.858868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1272/2559] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:20:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:20:04.017497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27ef0f-1295-408e-81d5-23b4372c88ba service glance] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:20:04.021813 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ad27ef0f-1295-408e-81d5-23b4372c88ba service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:04.067456 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:20:04.163087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27ef0f-1295-408e-81d5-23b4372c88ba service glance] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:20:04.169994 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a72d64-243f-41b6-b2e8-8f51890e82f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:20:04.170797 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a72d64-243f-41b6-b2e8-8f51890e82f1 service swift] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:04.171114 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a72d64-243f-41b6-b2e8-8f51890e82f1 service swift] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:04.171467 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a72d64-243f-41b6-b2e8-8f51890e82f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:04.182470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a72d64-243f-41b6-b2e8-8f51890e82f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:20:04.184964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a72d64-243f-41b6-b2e8-8f51890e82f1 service swift] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:20:04.201258 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1288/2560] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:20:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:20:13.799261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=56 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:20:14.196364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=23 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:20:17.433963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a50c4a-f5ba-43db-bd65-c098e7216834 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:17.434875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a50c4a-f5ba-43db-bd65-c098e7216834 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:17.434875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a50c4a-f5ba-43db-bd65-c098e7216834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:17.455603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-30a50c4a-f5ba-43db-bd65-c098e7216834 None None] MFA Rules not processed for user `a21183e645d74e4f9378c650e7916ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:20:17.456771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-30a50c4a-f5ba-43db-bd65-c098e7216834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:17.464686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1273/2561] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:20:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:20:17.554693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a72d64-243f-41b6-b2e8-8f51890e82f1 service swift] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:20:17.567079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a72d64-243f-41b6-b2e8-8f51890e82f1 service swift] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:20:17.571353 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb7e10-89a0-4a8b-bf49-5c1e4650ba66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:20:17.572212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb7e10-89a0-4a8b-bf49-5c1e4650ba66 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:17.572594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb7e10-89a0-4a8b-bf49-5c1e4650ba66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:17.572962 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb7e10-89a0-4a8b-bf49-5c1e4650ba66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:17.575096 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-50cb7e10-89a0-4a8b-bf49-5c1e4650ba66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:17.673127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb7e10-89a0-4a8b-bf49-5c1e4650ba66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:20:17.676676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb7e10-89a0-4a8b-bf49-5c1e4650ba66 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:20:17.696752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1289/2562] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:20:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:20:21.456576 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3174f59-6012-42e8-8e7b-a33fbb549bfb None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:21.458425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3174f59-6012-42e8-8e7b-a33fbb549bfb None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:21.459013 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3174f59-6012-42e8-8e7b-a33fbb549bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:21.487505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-b3174f59-6012-42e8-8e7b-a33fbb549bfb None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:20:21.489142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b3174f59-6012-42e8-8e7b-a33fbb549bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:21.513278 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1274/2563] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:20:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:20:21.521381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb7e10-89a0-4a8b-bf49-5c1e4650ba66 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:20:21.538319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb7e10-89a0-4a8b-bf49-5c1e4650ba66 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:20:21.546345 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deefb97-d2cd-43fd-b2e4-23404b11ae38 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:20:21.547314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deefb97-d2cd-43fd-b2e4-23404b11ae38 service swift] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:21.547774 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deefb97-d2cd-43fd-b2e4-23404b11ae38 service swift] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:21.548353 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deefb97-d2cd-43fd-b2e4-23404b11ae38 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:21.561275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deefb97-d2cd-43fd-b2e4-23404b11ae38 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:20:21.563696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deefb97-d2cd-43fd-b2e4-23404b11ae38 service swift] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:20:21.576516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1290/2564] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:20:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:20:31.481398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=6 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:20:31.571953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=36 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:20:46.421206 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84aa35-8057-47a4-a825-b9a2f5e9ecd3 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:46.421728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84aa35-8057-47a4-a825-b9a2f5e9ecd3 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:46.421938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84aa35-8057-47a4-a825-b9a2f5e9ecd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:46.442569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-6c84aa35-8057-47a4-a825-b9a2f5e9ecd3 None None] MFA Rules not processed for user `6d5a0cf37e3b4d3084a178f83642c144`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:20:46.443772 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6c84aa35-8057-47a4-a825-b9a2f5e9ecd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:46.498725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1275/2565] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:20:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:20:56.491632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=25 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:07.165065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deefb97-d2cd-43fd-b2e4-23404b11ae38 service swift] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:07.176549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deefb97-d2cd-43fd-b2e4-23404b11ae38 service swift] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:07.180319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb29b5e-0e36-4171-acf5-f57358dd7249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:07.180935 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb29b5e-0e36-4171-acf5-f57358dd7249 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:07.181207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb29b5e-0e36-4171-acf5-f57358dd7249 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:07.181477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb29b5e-0e36-4171-acf5-f57358dd7249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:07.191802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb29b5e-0e36-4171-acf5-f57358dd7249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:07.193295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb29b5e-0e36-4171-acf5-f57358dd7249 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:07.203666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1291/2566] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:21:08.998289 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84aa35-8057-47a4-a825-b9a2f5e9ecd3 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:09.000231 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6c84aa35-8057-47a4-a825-b9a2f5e9ecd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:09.083349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84aa35-8057-47a4-a825-b9a2f5e9ecd3 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:09.088532 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0537173-a4cb-4c7f-bf15-1b0d25d43d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:09.089209 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0537173-a4cb-4c7f-bf15-1b0d25d43d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:09.089485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0537173-a4cb-4c7f-bf15-1b0d25d43d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:09.089748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0537173-a4cb-4c7f-bf15-1b0d25d43d14 admin admin] PATH_INFO: `/v3/users/64b579837fb4434282224eb7b83e57ab` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:09.092841 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0537173-a4cb-4c7f-bf15-1b0d25d43d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b579837fb4434282224eb7b83e57ab)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:09.093257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0537173-a4cb-4c7f-bf15-1b0d25d43d14 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:09.162786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-a0537173-a4cb-4c7f-bf15-1b0d25d43d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b579837fb4434282224eb7b83e57ab'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:09.174398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-a0537173-a4cb-4c7f-bf15-1b0d25d43d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b579837fb4434282224eb7b83e57ab'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:09.175819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-a0537173-a4cb-4c7f-bf15-1b0d25d43d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b579837fb4434282224eb7b83e57ab'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:09.176883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-a0537173-a4cb-4c7f-bf15-1b0d25d43d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b579837fb4434282224eb7b83e57ab'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:09.202938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-a0537173-a4cb-4c7f-bf15-1b0d25d43d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b579837fb4434282224eb7b83e57ab'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:09.243651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1276/2567] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:21:08 2022] DELETE /identity/v3/users/64b579837fb4434282224eb7b83e57ab => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:21:09.582054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb29b5e-0e36-4171-acf5-f57358dd7249 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:09.599174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb29b5e-0e36-4171-acf5-f57358dd7249 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:09.746205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:09.747341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:09.747622 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:09.748100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] PATH_INFO: `/v3/projects/2815adc7e64040c8986b1bd3f4519343` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:09.749891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2815adc7e64040c8986b1bd3f4519343)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:09.751913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:09.858313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2815adc7e64040c8986b1bd3f4519343', 'request_id': 'req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:09.881563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2815adc7e64040c8986b1bd3f4519343', 'request_id': 'req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:09.900705 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2815adc7e64040c8986b1bd3f4519343', 'request_id': 'req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:09.941473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] The token cache is being invalidate because project 2815adc7e64040c8986b1bd3f4519343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:21:09.942702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:09.944399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:09.948454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1292/2568] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:21:09 2022] DELETE /identity/v3/projects/2815adc7e64040c8986b1bd3f4519343 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:21:19.233627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:19.927393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:21.209365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0f886-b5dc-45ee-8e8f-3bf68520f36b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:21.210507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0f886-b5dc-45ee-8e8f-3bf68520f36b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:21.210810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0f886-b5dc-45ee-8e8f-3bf68520f36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:21.230822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-03e0f886-b5dc-45ee-8e8f-3bf68520f36b None None] MFA Rules not processed for user `d5bd0cb311db42d7882a4cf90b0d85e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:21:21.232358 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-03e0f886-b5dc-45ee-8e8f-3bf68520f36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:21.289884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1277/2569] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:21:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:21:26.665498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:26.667018 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:26.745351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5e59c-6b4a-491a-bab2-621cc1de9f7e admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:26.749310 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6682396-c0c6-4764-ba5f-101e6294966b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:26.749974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6682396-c0c6-4764-ba5f-101e6294966b service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:26.750193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6682396-c0c6-4764-ba5f-101e6294966b service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:26.750459 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6682396-c0c6-4764-ba5f-101e6294966b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:26.752373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e6682396-c0c6-4764-ba5f-101e6294966b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:26.844013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6682396-c0c6-4764-ba5f-101e6294966b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:26.845470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6682396-c0c6-4764-ba5f-101e6294966b service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:26.858349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1293/2570] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:21:28.038449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0f886-b5dc-45ee-8e8f-3bf68520f36b None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:28.039875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-03e0f886-b5dc-45ee-8e8f-3bf68520f36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:28.052364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0f886-b5dc-45ee-8e8f-3bf68520f36b None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:28.055983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151149-4768-471a-b63a-b5c86ce24d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:28.056621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151149-4768-471a-b63a-b5c86ce24d24 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:28.056944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151149-4768-471a-b63a-b5c86ce24d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:28.057166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151149-4768-471a-b63a-b5c86ce24d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:28.058991 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1a151149-4768-471a-b63a-b5c86ce24d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:28.075412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a151149-4768-471a-b63a-b5c86ce24d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:28.076941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a151149-4768-471a-b63a-b5c86ce24d24 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:28.089426 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1278/2571] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:21:34.019867 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6682396-c0c6-4764-ba5f-101e6294966b service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:34.021293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e6682396-c0c6-4764-ba5f-101e6294966b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:34.035644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6682396-c0c6-4764-ba5f-101e6294966b service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:34.041197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191677d-97bd-4abc-afd1-e37f6d5134ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:34.041812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191677d-97bd-4abc-afd1-e37f6d5134ef service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:34.042076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191677d-97bd-4abc-afd1-e37f6d5134ef service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:34.042343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191677d-97bd-4abc-afd1-e37f6d5134ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:34.044038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d191677d-97bd-4abc-afd1-e37f6d5134ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:34.055254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191677d-97bd-4abc-afd1-e37f6d5134ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:34.056865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191677d-97bd-4abc-afd1-e37f6d5134ef service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:34.066194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1294/2572] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:21:35.336260 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151149-4768-471a-b63a-b5c86ce24d24 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:35.345128 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151149-4768-471a-b63a-b5c86ce24d24 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:35.348918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee41d2-ad81-4f04-b164-cbf8187d4f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:35.349513 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee41d2-ad81-4f04-b164-cbf8187d4f23 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:35.349785 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee41d2-ad81-4f04-b164-cbf8187d4f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:35.350048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee41d2-ad81-4f04-b164-cbf8187d4f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:35.351734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0aee41d2-ad81-4f04-b164-cbf8187d4f23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:35.418467 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee41d2-ad81-4f04-b164-cbf8187d4f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:35.419954 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee41d2-ad81-4f04-b164-cbf8187d4f23 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:35.429346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1279/2573] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:35 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:21:44.061786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=60 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:45.425019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=59 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:52.136108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191677d-97bd-4abc-afd1-e37f6d5134ef service nova] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:52.150048 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191677d-97bd-4abc-afd1-e37f6d5134ef service nova] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:52.153693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716f55c-c4ce-4484-8cf8-160ec99b395b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:52.154295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716f55c-c4ce-4484-8cf8-160ec99b395b service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:52.154574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716f55c-c4ce-4484-8cf8-160ec99b395b service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:52.154846 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716f55c-c4ce-4484-8cf8-160ec99b395b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:52.156626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a716f55c-c4ce-4484-8cf8-160ec99b395b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:52.222077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716f55c-c4ce-4484-8cf8-160ec99b395b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:52.223538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716f55c-c4ce-4484-8cf8-160ec99b395b service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:52.233562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1295/2574] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:21:55.526384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad02b77-1a30-437b-acd6-13dcaa5c2dd5 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:55.526934 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad02b77-1a30-437b-acd6-13dcaa5c2dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:55.527054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad02b77-1a30-437b-acd6-13dcaa5c2dd5 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:55.528656 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1280/2575] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:21:55.557128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d703029-a62d-4078-bcbb-d2e55527bb9b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:55.557641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d703029-a62d-4078-bcbb-d2e55527bb9b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:55.557641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d703029-a62d-4078-bcbb-d2e55527bb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:55.578289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-0d703029-a62d-4078-bcbb-d2e55527bb9b None None] MFA Rules not processed for user `08f34295f19b4d6ea405f1e8c8190db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:21:55.579544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0d703029-a62d-4078-bcbb-d2e55527bb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:55.591982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad02b77-1a30-437b-acd6-13dcaa5c2dd5 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:55.594059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-bad02b77-1a30-437b-acd6-13dcaa5c2dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:55.615899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad02b77-1a30-437b-acd6-13dcaa5c2dd5 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:55.620676 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:55.621668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:55.622010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:55.622349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] PATH_INFO: `/v3/users/6d5a0cf37e3b4d3084a178f83642c144` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:55.623548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d5a0cf37e3b4d3084a178f83642c144)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:55.625588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:55.656126 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1296/2576] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:21:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:21:55.670338 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b089094-36f0-4b74-97b5-1b6755447afd None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:55.670621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b089094-36f0-4b74-97b5-1b6755447afd None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:55.670942 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b089094-36f0-4b74-97b5-1b6755447afd None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:55.672650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1297/2577] 10.222.0.44 () {54 vars in 961 bytes} [Tue Aug 30 01:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:21:55.678823 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f16456-a061-474b-a96c-2369667ae420 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:55.679111 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f16456-a061-474b-a96c-2369667ae420 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:55.679495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f16456-a061-474b-a96c-2369667ae420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:55.681003 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d5a0cf37e3b4d3084a178f83642c144'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:55.691650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d5a0cf37e3b4d3084a178f83642c144'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:55.693761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d5a0cf37e3b4d3084a178f83642c144'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:55.694550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d5a0cf37e3b4d3084a178f83642c144'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:55.702897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-29f16456-a061-474b-a96c-2369667ae420 None None] MFA Rules not processed for user `08f34295f19b4d6ea405f1e8c8190db5`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:21:55.704384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-29f16456-a061-474b-a96c-2369667ae420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:55.714092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d5a0cf37e3b4d3084a178f83642c144'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:55.732908 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1298/2578] 10.222.0.44 () {58 vars in 1058 bytes} [Tue Aug 30 01:21:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:21:55.734052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f16456-a061-474b-a96c-2369667ae420 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:55.742203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1281/2579] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:21:55 2022] DELETE /identity/v3/users/6d5a0cf37e3b4d3084a178f83642c144 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:21:55.748371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f16456-a061-474b-a96c-2369667ae420 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:55.812109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968669c-251f-477b-9815-0620da93b170 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08f34295f19b4d6ea405f1e8c8190db5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:55.812653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968669c-251f-477b-9815-0620da93b170 service placement] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:55.812932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968669c-251f-477b-9815-0620da93b170 service placement] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:55.813204 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968669c-251f-477b-9815-0620da93b170 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:55.815213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1968669c-251f-477b-9815-0620da93b170 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:55.826527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968669c-251f-477b-9815-0620da93b170 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:55.828237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968669c-251f-477b-9815-0620da93b170 service placement] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:55.838632 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1299/2580] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:21:55.920589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:55.922078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:55.991107 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66f4a7-5b23-496b-85dc-517c3e2098ca admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:55.995752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:55.996670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:55.996948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:55.997220 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8 admin admin] PATH_INFO: `/v3/projects/adf47975f465467498be24ebe2c81ad7` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:55.998360 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf47975f465467498be24ebe2c81ad7)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:56.000332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:56.048414 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf47975f465467498be24ebe2c81ad7', 'request_id': 'req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:56.055317 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf47975f465467498be24ebe2c81ad7', 'request_id': 'req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:56.060988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf47975f465467498be24ebe2c81ad7', 'request_id': 'req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:56.074907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8 admin admin] The token cache is being invalidate because project adf47975f465467498be24ebe2c81ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:21:56.075171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:56.075782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-172cce87-afa1-4cd1-a4f2-b95013c2e6b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:21:56.077338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1282/2581] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:21:55 2022] DELETE /identity/v3/projects/adf47975f465467498be24ebe2c81ad7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:21:56.083188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfd90f-225e-47ae-8543-a1a96c6e720d None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:56.083469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfd90f-225e-47ae-8543-a1a96c6e720d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:56.083742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfd90f-225e-47ae-8543-a1a96c6e720d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:56.105069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-cfdfd90f-225e-47ae-8543-a1a96c6e720d None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:21:56.106260 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-cfdfd90f-225e-47ae-8543-a1a96c6e720d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:56.114970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:56.171130 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1300/2582] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:21:56.179098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4f6ba-7e2d-4553-b2b8-828e525d7852 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:56.179381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4f6ba-7e2d-4553-b2b8-828e525d7852 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:56.179655 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4f6ba-7e2d-4553-b2b8-828e525d7852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:56.187345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:56.324664 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-62d4f6ba-7e2d-4553-b2b8-828e525d7852 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:21:56.325952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-62d4f6ba-7e2d-4553-b2b8-828e525d7852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:56.335600 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1283/2583] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:21:56.339747 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfd90f-225e-47ae-8543-a1a96c6e720d None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:56.350541 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfd90f-225e-47ae-8543-a1a96c6e720d None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:56.355137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4667a-7062-490d-a662-4a19352b4c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:56.355747 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4667a-7062-490d-a662-4a19352b4c48 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:56.356051 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4667a-7062-490d-a662-4a19352b4c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:56.356327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4667a-7062-490d-a662-4a19352b4c48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:56.357003 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4667a-7062-490d-a662-4a19352b4c48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:21:56.357308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4667a-7062-490d-a662-4a19352b4c48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:56.358913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4667a-7062-490d-a662-4a19352b4c48 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:56.376397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1301/2584] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:21:56.377938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4f6ba-7e2d-4553-b2b8-828e525d7852 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:56.387099 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4f6ba-7e2d-4553-b2b8-828e525d7852 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:56.391660 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102992b7-eaea-4772-8119-06c84f473bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:56.392286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102992b7-eaea-4772-8119-06c84f473bfb admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:56.392560 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102992b7-eaea-4772-8119-06c84f473bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:56.392900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102992b7-eaea-4772-8119-06c84f473bfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:56.393531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102992b7-eaea-4772-8119-06c84f473bfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:56.395402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102992b7-eaea-4772-8119-06c84f473bfb admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:56.428970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1284/2585] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:21:56 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:21:56.433350 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4667a-7062-490d-a662-4a19352b4c48 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:56.443905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4667a-7062-490d-a662-4a19352b4c48 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:56.501891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3e28c-1c72-476f-be52-ca56ea85213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:56.502663 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3e28c-1c72-476f-be52-ca56ea85213f admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:56.502944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3e28c-1c72-476f-be52-ca56ea85213f admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:56.503215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3e28c-1c72-476f-be52-ca56ea85213f admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:56.503855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3e28c-1c72-476f-be52-ca56ea85213f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:56.505809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3e28c-1c72-476f-be52-ca56ea85213f admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:56.527084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1302/2586] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:21:56 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:21:56.531200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102992b7-eaea-4772-8119-06c84f473bfb admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:56.543671 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102992b7-eaea-4772-8119-06c84f473bfb admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:56.550495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c990d9-7b46-418f-a770-0a5b5498e05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:56.551134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c990d9-7b46-418f-a770-0a5b5498e05b admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:56.551405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c990d9-7b46-418f-a770-0a5b5498e05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:56.551668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c990d9-7b46-418f-a770-0a5b5498e05b admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:56.552339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c990d9-7b46-418f-a770-0a5b5498e05b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:56.553907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c990d9-7b46-418f-a770-0a5b5498e05b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:56.564072 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1285/2587] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:21:56.568378 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3e28c-1c72-476f-be52-ca56ea85213f admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:56.578408 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3e28c-1c72-476f-be52-ca56ea85213f admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:56.583370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a321c-0f61-4eb2-b856-cd123fd9dc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:56.583957 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a321c-0f61-4eb2-b856-cd123fd9dc94 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:56.584264 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a321c-0f61-4eb2-b856-cd123fd9dc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:56.584531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a321c-0f61-4eb2-b856-cd123fd9dc94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:56.585165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a321c-0f61-4eb2-b856-cd123fd9dc94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:56.586742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a321c-0f61-4eb2-b856-cd123fd9dc94 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:56.597719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1303/2588] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:21:56.602179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c990d9-7b46-418f-a770-0a5b5498e05b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:56.613060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c990d9-7b46-418f-a770-0a5b5498e05b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:56.617822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba345a-f195-4b6d-8fdb-58563bf30b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:56.618566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba345a-f195-4b6d-8fdb-58563bf30b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:56.618956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba345a-f195-4b6d-8fdb-58563bf30b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:56.619234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba345a-f195-4b6d-8fdb-58563bf30b19 admin admin] PATH_INFO: `/v3/projects/5d08b735ca4249309f4ecd40cbacd537/users/22bf08f4cdb24731bf00c7962bf81c40/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:56.635777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba345a-f195-4b6d-8fdb-58563bf30b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d08b735ca4249309f4ecd40cbacd537, user_id=22bf08f4cdb24731bf00c7962bf81c40, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:56.638412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba345a-f195-4b6d-8fdb-58563bf30b19 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:56.652470 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1286/2589] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:21:56 2022] PUT /identity/v3/projects/5d08b735ca4249309f4ecd40cbacd537/users/22bf08f4cdb24731bf00c7962bf81c40/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:21:56.657562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d5148-becc-469e-bfd2-067c389d4ac2 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:56.657845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d5148-becc-469e-bfd2-067c389d4ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:56.658115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d5148-becc-469e-bfd2-067c389d4ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:56.679103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b59d5148-becc-469e-bfd2-067c389d4ac2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:21:56.680327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b59d5148-becc-469e-bfd2-067c389d4ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:56.740949 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1304/2590] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:21:56.751125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba345a-f195-4b6d-8fdb-58563bf30b19 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:56.753151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-bfba345a-f195-4b6d-8fdb-58563bf30b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:56.810942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba345a-f195-4b6d-8fdb-58563bf30b19 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:56.814543 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56928a-8fa1-43fc-96ec-1cdd932b17ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:56.815158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56928a-8fa1-43fc-96ec-1cdd932b17ca service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:56.815430 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56928a-8fa1-43fc-96ec-1cdd932b17ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:56.815698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56928a-8fa1-43fc-96ec-1cdd932b17ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:56.824933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56928a-8fa1-43fc-96ec-1cdd932b17ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:56.826379 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56928a-8fa1-43fc-96ec-1cdd932b17ca service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:56.836283 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1287/2591] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:22:00.871787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9b6a6-7e21-4dc7-b295-04eabba215fa None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:00.872336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9b6a6-7e21-4dc7-b295-04eabba215fa None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:00.872435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9b6a6-7e21-4dc7-b295-04eabba215fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:00.903027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7ec9b6a6-7e21-4dc7-b295-04eabba215fa None None] MFA Rules not processed for user `22bf08f4cdb24731bf00c7962bf81c40`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:22:00.904328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7ec9b6a6-7e21-4dc7-b295-04eabba215fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:00.913941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:01.001353 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1305/2592] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:22:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:22:01.020819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56928a-8fa1-43fc-96ec-1cdd932b17ca service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:01.022329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-aa56928a-8fa1-43fc-96ec-1cdd932b17ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:01.106200 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56928a-8fa1-43fc-96ec-1cdd932b17ca service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:01.112474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2810b-906a-469f-9948-887251b79d8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:01.113059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2810b-906a-469f-9948-887251b79d8b service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:01.113332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2810b-906a-469f-9948-887251b79d8b service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:01.113600 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2810b-906a-469f-9948-887251b79d8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:01.122438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2810b-906a-469f-9948-887251b79d8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:01.124169 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2810b-906a-469f-9948-887251b79d8b service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:01.134851 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1288/2593] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:22:02.661579 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a03f16-24ea-4d6d-83a8-05bc8c15e93a None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:02.662069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a03f16-24ea-4d6d-83a8-05bc8c15e93a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:02.662172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a03f16-24ea-4d6d-83a8-05bc8c15e93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:02.682530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-00a03f16-24ea-4d6d-83a8-05bc8c15e93a None None] MFA Rules not processed for user `22bf08f4cdb24731bf00c7962bf81c40`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:22:02.683726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-00a03f16-24ea-4d6d-83a8-05bc8c15e93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:02.691508 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1306/2594] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:22:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:22:03.017029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2810b-906a-469f-9948-887251b79d8b service nova] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:03.018636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-52c2810b-906a-469f-9948-887251b79d8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:03.030404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2810b-906a-469f-9948-887251b79d8b service nova] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:03.035041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab942da1-151b-4153-b812-d3206ab4299f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:03.035640 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab942da1-151b-4153-b812-d3206ab4299f service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:03.035911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab942da1-151b-4153-b812-d3206ab4299f service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:03.036213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab942da1-151b-4153-b812-d3206ab4299f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:03.037915 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ab942da1-151b-4153-b812-d3206ab4299f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:03.047643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab942da1-151b-4153-b812-d3206ab4299f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:03.049736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab942da1-151b-4153-b812-d3206ab4299f service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:03.057730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1289/2595] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:22:03.394744 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a03f16-24ea-4d6d-83a8-05bc8c15e93a None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:03.396617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-00a03f16-24ea-4d6d-83a8-05bc8c15e93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:03.485105 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a03f16-24ea-4d6d-83a8-05bc8c15e93a None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:03.490921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cbf0b-2740-407d-85f7-25674f6b8414 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08f34295f19b4d6ea405f1e8c8190db5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:03.491568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cbf0b-2740-407d-85f7-25674f6b8414 service placement] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:03.491842 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cbf0b-2740-407d-85f7-25674f6b8414 service placement] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:03.492709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cbf0b-2740-407d-85f7-25674f6b8414 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:03.494614 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0b1cbf0b-2740-407d-85f7-25674f6b8414 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:03.508555 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1cbf0b-2740-407d-85f7-25674f6b8414 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:03.510146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1cbf0b-2740-407d-85f7-25674f6b8414 service placement] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:03.521737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1307/2596] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:22:05.599816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab942da1-151b-4153-b812-d3206ab4299f service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:05.608779 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab942da1-151b-4153-b812-d3206ab4299f service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:05.615143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5948e2c-caab-45ad-980c-797b47c48170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:05.616029 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5948e2c-caab-45ad-980c-797b47c48170 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:05.616478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5948e2c-caab-45ad-980c-797b47c48170 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:05.616744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5948e2c-caab-45ad-980c-797b47c48170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:05.618618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e5948e2c-caab-45ad-980c-797b47c48170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:05.631188 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5948e2c-caab-45ad-980c-797b47c48170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:05.632872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5948e2c-caab-45ad-980c-797b47c48170 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:05.637638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:05.643323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1290/2597] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:22:06.302858 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cbf0b-2740-407d-85f7-25674f6b8414 service placement] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:06.304785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0b1cbf0b-2740-407d-85f7-25674f6b8414 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:06.321452 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cbf0b-2740-407d-85f7-25674f6b8414 service placement] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:06.328689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28209f50-d8f7-49ea-8344-7259a8136ae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:06.329715 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28209f50-d8f7-49ea-8344-7259a8136ae1 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:06.330096 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28209f50-d8f7-49ea-8344-7259a8136ae1 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:06.330954 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28209f50-d8f7-49ea-8344-7259a8136ae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:06.333134 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-28209f50-d8f7-49ea-8344-7259a8136ae1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:06.347485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28209f50-d8f7-49ea-8344-7259a8136ae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:06.349865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28209f50-d8f7-49ea-8344-7259a8136ae1 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:06.363727 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1308/2598] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:22:12.867080 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5948e2c-caab-45ad-980c-797b47c48170 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:12.885123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5948e2c-caab-45ad-980c-797b47c48170 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:12.890663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca48889-01a4-4c61-a8e1-9379c9626e70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:12.891308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca48889-01a4-4c61-a8e1-9379c9626e70 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:12.892156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca48889-01a4-4c61-a8e1-9379c9626e70 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:12.892613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca48889-01a4-4c61-a8e1-9379c9626e70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:12.894869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-eca48889-01a4-4c61-a8e1-9379c9626e70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:12.913832 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca48889-01a4-4c61-a8e1-9379c9626e70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:12.915428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca48889-01a4-4c61-a8e1-9379c9626e70 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:12.929056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1291/2599] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:22:16.357785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=64 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:22.923434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=6 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:34.201491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79708c6c-2478-4abd-94d3-3b51ce4339ff None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:34.202123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79708c6c-2478-4abd-94d3-3b51ce4339ff None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:34.202123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79708c6c-2478-4abd-94d3-3b51ce4339ff None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:34.204060 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1309/2600] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:22:34.211961 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ac9a7-8d7a-4038-b406-6e36ae5923d0 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:34.212499 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ac9a7-8d7a-4038-b406-6e36ae5923d0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:34.212876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ac9a7-8d7a-4038-b406-6e36ae5923d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:34.232692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-332ac9a7-8d7a-4038-b406-6e36ae5923d0 None None] MFA Rules not processed for user `86cea2f9238b42e0a7cb26bac494a8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:22:34.234502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-332ac9a7-8d7a-4038-b406-6e36ae5923d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:34.241250 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1292/2601] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:22:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:22:34.254688 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79708c6c-2478-4abd-94d3-3b51ce4339ff None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:34.266168 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79708c6c-2478-4abd-94d3-3b51ce4339ff None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:34.269819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ea081-7501-4d53-9def-bca4cf33a06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:34.270578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ea081-7501-4d53-9def-bca4cf33a06a service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:34.270904 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ea081-7501-4d53-9def-bca4cf33a06a service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:34.271179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ea081-7501-4d53-9def-bca4cf33a06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:34.278659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ea081-7501-4d53-9def-bca4cf33a06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:34.280117 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ea081-7501-4d53-9def-bca4cf33a06a service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:34.288309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1310/2602] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:22:34.371361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0287f93-5c8b-40c1-b872-d09586715a29 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:34.371729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0287f93-5c8b-40c1-b872-d09586715a29 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:34.372120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0287f93-5c8b-40c1-b872-d09586715a29 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:34.373547 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1293/2603] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:22:34.379823 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57bf7a-a0af-4475-939b-cf4092660315 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:34.380391 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57bf7a-a0af-4475-939b-cf4092660315 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:34.380700 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57bf7a-a0af-4475-939b-cf4092660315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:34.402061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ac57bf7a-a0af-4475-939b-cf4092660315 None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:22:34.403265 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ac57bf7a-a0af-4475-939b-cf4092660315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:34.412836 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1311/2604] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:22:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:22:34.424352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0287f93-5c8b-40c1-b872-d09586715a29 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:34.435658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0287f93-5c8b-40c1-b872-d09586715a29 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:34.440093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f329371-3a13-4e11-9c80-b59ad3f50de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:34.440688 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f329371-3a13-4e11-9c80-b59ad3f50de9 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:34.440965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f329371-3a13-4e11-9c80-b59ad3f50de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:34.441228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f329371-3a13-4e11-9c80-b59ad3f50de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:34.451645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f329371-3a13-4e11-9c80-b59ad3f50de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:34.453103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f329371-3a13-4e11-9c80-b59ad3f50de9 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:34.464178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1294/2605] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:22:42.092974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4a52b-4663-41f3-86cd-66208180788e None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:42.093490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4a52b-4663-41f3-86cd-66208180788e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:42.093490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4a52b-4663-41f3-86cd-66208180788e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:42.113504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e3e4a52b-4663-41f3-86cd-66208180788e None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:22:42.114696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e3e4a52b-4663-41f3-86cd-66208180788e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:42.173349 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1312/2606] 10.222.0.44 () {58 vars in 967 bytes} [Tue Aug 30 01:22:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:22:42.219033 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f329371-3a13-4e11-9c80-b59ad3f50de9 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:42.220549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4f329371-3a13-4e11-9c80-b59ad3f50de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:42.293027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f329371-3a13-4e11-9c80-b59ad3f50de9 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:42.298876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e875ec-95ea-45ed-b12a-2d43e268afdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:42.299498 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e875ec-95ea-45ed-b12a-2d43e268afdc service swift] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:42.299775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e875ec-95ea-45ed-b12a-2d43e268afdc service swift] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:42.300077 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e875ec-95ea-45ed-b12a-2d43e268afdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:42.308965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e875ec-95ea-45ed-b12a-2d43e268afdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:42.310431 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e875ec-95ea-45ed-b12a-2d43e268afdc service swift] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:42.320617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1295/2607] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:22:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:22:44.707044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4a52b-4663-41f3-86cd-66208180788e None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:44.734862 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4a52b-4663-41f3-86cd-66208180788e None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:44.749927 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cf111-77b0-4407-b742-1fc8c3961715 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:44.751474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cf111-77b0-4407-b742-1fc8c3961715 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:44.752323 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cf111-77b0-4407-b742-1fc8c3961715 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:44.753001 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cf111-77b0-4407-b742-1fc8c3961715 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:44.756184 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-601cf111-77b0-4407-b742-1fc8c3961715 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:44.783045 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cf111-77b0-4407-b742-1fc8c3961715 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:44.786992 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cf111-77b0-4407-b742-1fc8c3961715 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:44.833263 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1313/2608] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:22:45.229916 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb875ff-72f8-47cb-afdb-b05ad9839875 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:45.230839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb875ff-72f8-47cb-afdb-b05ad9839875 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:45.231315 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb875ff-72f8-47cb-afdb-b05ad9839875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:45.271794 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-6bb875ff-72f8-47cb-afdb-b05ad9839875 None None] MFA Rules not processed for user `2839103f52324a1ab1ebd43fe230addc`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:22:45.273394 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6bb875ff-72f8-47cb-afdb-b05ad9839875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:45.423258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1296/2609] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:22:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:22:51.372662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cf111-77b0-4407-b742-1fc8c3961715 service nova] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:51.392759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cf111-77b0-4407-b742-1fc8c3961715 service nova] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:51.397094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db350003-f207-4548-bdda-6d0fa6a98b96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:51.398141 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db350003-f207-4548-bdda-6d0fa6a98b96 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:51.398781 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db350003-f207-4548-bdda-6d0fa6a98b96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:51.399288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db350003-f207-4548-bdda-6d0fa6a98b96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:51.402017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-db350003-f207-4548-bdda-6d0fa6a98b96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:51.418996 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db350003-f207-4548-bdda-6d0fa6a98b96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:51.422186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db350003-f207-4548-bdda-6d0fa6a98b96 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:51.441984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1314/2610] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:22:52.132471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb875ff-72f8-47cb-afdb-b05ad9839875 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:52.134047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6bb875ff-72f8-47cb-afdb-b05ad9839875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:52.149142 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb875ff-72f8-47cb-afdb-b05ad9839875 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:52.154130 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:52.154877 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:52.155756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:52.156694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] PATH_INFO: `/v3/users/a21183e645d74e4f9378c650e7916ba8` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:52.157797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a21183e645d74e4f9378c650e7916ba8)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:52.159931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:52.222802 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a21183e645d74e4f9378c650e7916ba8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.235569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a21183e645d74e4f9378c650e7916ba8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.236378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a21183e645d74e4f9378c650e7916ba8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.237080 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a21183e645d74e4f9378c650e7916ba8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.251299 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:52.253198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a21183e645d74e4f9378c650e7916ba8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.275625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1297/2611] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:22:52 2022] DELETE /identity/v3/users/a21183e645d74e4f9378c650e7916ba8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:22:52.498530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db350003-f207-4548-bdda-6d0fa6a98b96 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:52.510375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db350003-f207-4548-bdda-6d0fa6a98b96 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:52.575277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:52.575924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:52.576273 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:52.576514 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] PATH_INFO: `/v3/projects/91f4e4df4e2744bfb9fcd84431ed567e` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:52.577790 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f4e4df4e2744bfb9fcd84431ed567e)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:52.579755 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:52.632014 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f4e4df4e2744bfb9fcd84431ed567e', 'request_id': 'req-062b1163-433e-436d-b76e-a58568eacf14'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.640594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f4e4df4e2744bfb9fcd84431ed567e', 'request_id': 'req-062b1163-433e-436d-b76e-a58568eacf14'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.645030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:52.647246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f4e4df4e2744bfb9fcd84431ed567e', 'request_id': 'req-062b1163-433e-436d-b76e-a58568eacf14'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.674442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] The token cache is being invalidate because project 91f4e4df4e2744bfb9fcd84431ed567e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:22:52.674745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.675743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.677327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1315/2612] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:22:52 2022] DELETE /identity/v3/projects/91f4e4df4e2744bfb9fcd84431ed567e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:22:52.681389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:52.682836 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:52.699463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9598a-999b-4954-bd04-7aa305cd4267 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:52.760639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:52.761390 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:52.761734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:52.762014 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] PATH_INFO: `/v3/users/0a9b2de11b724397ad569ee7f5372ff4` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:52.763155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a9b2de11b724397ad569ee7f5372ff4)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:52.765864 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:52.815213 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a9b2de11b724397ad569ee7f5372ff4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.822808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a9b2de11b724397ad569ee7f5372ff4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.823845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a9b2de11b724397ad569ee7f5372ff4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.824124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a9b2de11b724397ad569ee7f5372ff4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.838177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a9b2de11b724397ad569ee7f5372ff4'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:52.853917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1298/2613] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:22:52 2022] DELETE /identity/v3/users/0a9b2de11b724397ad569ee7f5372ff4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:22:53.215615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:53.218071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:53.246381 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b1163-433e-436d-b76e-a58568eacf14 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:53.413405 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:53.414252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:53.414896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:53.415358 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] PATH_INFO: `/v3/projects/0ce5293d77734c08bdae2dd3a01ca285` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:53.417760 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce5293d77734c08bdae2dd3a01ca285)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:53.420559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:53.509304 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce5293d77734c08bdae2dd3a01ca285', 'request_id': 'req-5e332def-da2f-420e-98b4-80b32f6e332c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:53.518230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce5293d77734c08bdae2dd3a01ca285', 'request_id': 'req-5e332def-da2f-420e-98b4-80b32f6e332c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:53.524571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce5293d77734c08bdae2dd3a01ca285', 'request_id': 'req-5e332def-da2f-420e-98b4-80b32f6e332c'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:53.536984 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] The token cache is being invalidate because project 0ce5293d77734c08bdae2dd3a01ca285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:22:53.537379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:53.538107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:53.539729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1316/2614] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:22:53 2022] DELETE /identity/v3/projects/0ce5293d77734c08bdae2dd3a01ca285 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:22:55.967963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:55.969745 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:55.983400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f7c2b-ea2e-46b5-b6d9-de9a5067fa0b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:56.057417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016dc75-d229-4302-8fed-45c316db28d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:56.058062 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016dc75-d229-4302-8fed-45c316db28d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:56.058457 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016dc75-d229-4302-8fed-45c316db28d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:56.058731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016dc75-d229-4302-8fed-45c316db28d4 admin admin] PATH_INFO: `/v3/users/d5bd0cb311db42d7882a4cf90b0d85e8` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:56.060135 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016dc75-d229-4302-8fed-45c316db28d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5bd0cb311db42d7882a4cf90b0d85e8)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:56.062837 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016dc75-d229-4302-8fed-45c316db28d4 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:56.069834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:56.121921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e016dc75-d229-4302-8fed-45c316db28d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5bd0cb311db42d7882a4cf90b0d85e8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:56.129829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e016dc75-d229-4302-8fed-45c316db28d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5bd0cb311db42d7882a4cf90b0d85e8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:56.130660 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e016dc75-d229-4302-8fed-45c316db28d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5bd0cb311db42d7882a4cf90b0d85e8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:56.131297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e016dc75-d229-4302-8fed-45c316db28d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5bd0cb311db42d7882a4cf90b0d85e8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:56.144726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-e016dc75-d229-4302-8fed-45c316db28d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5bd0cb311db42d7882a4cf90b0d85e8'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:56.160416 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1299/2615] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:22:55 2022] DELETE /identity/v3/users/d5bd0cb311db42d7882a4cf90b0d85e8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:22:56.341087 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:56.343871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:56.363780 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e332def-da2f-420e-98b4-80b32f6e332c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:56.427285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843df738-2f69-4175-9136-2a510aa78b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:56.427921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843df738-2f69-4175-9136-2a510aa78b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:56.428241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843df738-2f69-4175-9136-2a510aa78b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:56.428511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843df738-2f69-4175-9136-2a510aa78b92 admin admin] PATH_INFO: `/v3/projects/a69f2b2876ed47459a8b960ff679c724` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:56.429639 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843df738-2f69-4175-9136-2a510aa78b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a69f2b2876ed47459a8b960ff679c724)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:56.431548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843df738-2f69-4175-9136-2a510aa78b92 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:56.453953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:56.478759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-843df738-2f69-4175-9136-2a510aa78b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a69f2b2876ed47459a8b960ff679c724', 'request_id': 'req-843df738-2f69-4175-9136-2a510aa78b92'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:56.487373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-843df738-2f69-4175-9136-2a510aa78b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a69f2b2876ed47459a8b960ff679c724', 'request_id': 'req-843df738-2f69-4175-9136-2a510aa78b92'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:56.493220 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-843df738-2f69-4175-9136-2a510aa78b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a69f2b2876ed47459a8b960ff679c724', 'request_id': 'req-843df738-2f69-4175-9136-2a510aa78b92'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:56.504648 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-843df738-2f69-4175-9136-2a510aa78b92 admin admin] The token cache is being invalidate because project a69f2b2876ed47459a8b960ff679c724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:22:56.504934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-843df738-2f69-4175-9136-2a510aa78b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:56.505594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-843df738-2f69-4175-9136-2a510aa78b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:56.507092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1317/2616] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:22:56 2022] DELETE /identity/v3/projects/a69f2b2876ed47459a8b960ff679c724 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:22:56.512055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0629812-83d2-4c6d-9192-f3cfb40a9444 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:56.512341 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0629812-83d2-4c6d-9192-f3cfb40a9444 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:56.512611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0629812-83d2-4c6d-9192-f3cfb40a9444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:56.533296 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d0629812-83d2-4c6d-9192-f3cfb40a9444 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:22:56.534495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d0629812-83d2-4c6d-9192-f3cfb40a9444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:56.595343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1300/2617] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:22:56.603305 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ad6d7-8420-40d4-9b9a-e4bdbc4830ed None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:56.603587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ad6d7-8420-40d4-9b9a-e4bdbc4830ed None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:56.603865 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ad6d7-8420-40d4-9b9a-e4bdbc4830ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:56.625167 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-fd0ad6d7-8420-40d4-9b9a-e4bdbc4830ed None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:22:56.626375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-fd0ad6d7-8420-40d4-9b9a-e4bdbc4830ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:56.634360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1318/2618] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:22:56.638535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0629812-83d2-4c6d-9192-f3cfb40a9444 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:56.649528 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0629812-83d2-4c6d-9192-f3cfb40a9444 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:56.654219 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a0b4e-9fd8-42a8-aec4-660728c07123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:56.654882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a0b4e-9fd8-42a8-aec4-660728c07123 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:56.655654 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a0b4e-9fd8-42a8-aec4-660728c07123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:56.655923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a0b4e-9fd8-42a8-aec4-660728c07123 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:56.656601 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a0b4e-9fd8-42a8-aec4-660728c07123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:22:56.656890 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a0b4e-9fd8-42a8-aec4-660728c07123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:56.658729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a0b4e-9fd8-42a8-aec4-660728c07123 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:56.670278 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1301/2619] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:22:56.674900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ad6d7-8420-40d4-9b9a-e4bdbc4830ed None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:56.684313 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ad6d7-8420-40d4-9b9a-e4bdbc4830ed None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:56.690777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c80cd-ef15-41e6-94c7-677c9dd8fef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:56.691390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c80cd-ef15-41e6-94c7-677c9dd8fef6 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:56.691668 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c80cd-ef15-41e6-94c7-677c9dd8fef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:56.691938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c80cd-ef15-41e6-94c7-677c9dd8fef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:56.692613 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c80cd-ef15-41e6-94c7-677c9dd8fef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:56.694540 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c80cd-ef15-41e6-94c7-677c9dd8fef6 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:56.723869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1319/2620] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:22:56 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:22:56.728165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a0b4e-9fd8-42a8-aec4-660728c07123 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:56.738733 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a0b4e-9fd8-42a8-aec4-660728c07123 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:56.795008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ad2e5-a31d-43a3-95e6-dace64c46c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:56.796139 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ad2e5-a31d-43a3-95e6-dace64c46c4f admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:56.796419 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ad2e5-a31d-43a3-95e6-dace64c46c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:56.796717 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ad2e5-a31d-43a3-95e6-dace64c46c4f admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:56.797352 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ad2e5-a31d-43a3-95e6-dace64c46c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:56.799251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ad2e5-a31d-43a3-95e6-dace64c46c4f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:56.824465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1302/2621] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:22:56 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:22:56.828888 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c80cd-ef15-41e6-94c7-677c9dd8fef6 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:56.839733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c80cd-ef15-41e6-94c7-677c9dd8fef6 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:56.844583 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a5572-b8fa-4e11-b078-278f83fdb2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:56.845181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a5572-b8fa-4e11-b078-278f83fdb2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:56.845458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a5572-b8fa-4e11-b078-278f83fdb2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:56.845723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a5572-b8fa-4e11-b078-278f83fdb2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:56.846364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a5572-b8fa-4e11-b078-278f83fdb2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:56.847953 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a5572-b8fa-4e11-b078-278f83fdb2e4 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:56.858553 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1320/2622] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:22:56.862681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ad2e5-a31d-43a3-95e6-dace64c46c4f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:56.868032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:56.873109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ad2e5-a31d-43a3-95e6-dace64c46c4f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:56.880796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775aa542-e5d1-4e41-9a68-8981883b8ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:56.881407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775aa542-e5d1-4e41-9a68-8981883b8ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:56.881941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775aa542-e5d1-4e41-9a68-8981883b8ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:56.882307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775aa542-e5d1-4e41-9a68-8981883b8ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:56.882927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775aa542-e5d1-4e41-9a68-8981883b8ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:56.884842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775aa542-e5d1-4e41-9a68-8981883b8ce8 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:56.896038 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1303/2623] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:22:56.900414 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a5572-b8fa-4e11-b078-278f83fdb2e4 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:56.910702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a5572-b8fa-4e11-b078-278f83fdb2e4 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:56.915427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f884751-b2d4-422a-a38c-9495f162587b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:56.916166 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f884751-b2d4-422a-a38c-9495f162587b admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:56.916442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f884751-b2d4-422a-a38c-9495f162587b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:56.916708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f884751-b2d4-422a-a38c-9495f162587b admin admin] PATH_INFO: `/v3/projects/f5ddbf041c444253aea027770188fbaf/users/4fe36a87392c40379ac160ebaa3cd84c/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:56.932397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f884751-b2d4-422a-a38c-9495f162587b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ddbf041c444253aea027770188fbaf, user_id=4fe36a87392c40379ac160ebaa3cd84c, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:56.935050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f884751-b2d4-422a-a38c-9495f162587b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:56.949307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1321/2624] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:22:56 2022] PUT /identity/v3/projects/f5ddbf041c444253aea027770188fbaf/users/4fe36a87392c40379ac160ebaa3cd84c/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:22:56.954814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343af01-09f8-4012-8ce1-d054d719bee1 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:56.955099 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343af01-09f8-4012-8ce1-d054d719bee1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:56.955425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343af01-09f8-4012-8ce1-d054d719bee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:56.974538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-e343af01-09f8-4012-8ce1-d054d719bee1 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:22:56.975735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e343af01-09f8-4012-8ce1-d054d719bee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:57.038533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1304/2625] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:22:57.047913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f884751-b2d4-422a-a38c-9495f162587b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:57.050127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5f884751-b2d4-422a-a38c-9495f162587b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:57.118551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f884751-b2d4-422a-a38c-9495f162587b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:57.122248 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746fddc-c301-4418-9348-7a5214ce3ddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:57.122939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746fddc-c301-4418-9348-7a5214ce3ddd service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:57.123286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746fddc-c301-4418-9348-7a5214ce3ddd service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:57.123606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746fddc-c301-4418-9348-7a5214ce3ddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:57.133238 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746fddc-c301-4418-9348-7a5214ce3ddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:57.134676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746fddc-c301-4418-9348-7a5214ce3ddd service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:57.144781 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1322/2626] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:23:01.915211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420e685-de6b-4ba1-a64a-89ea6c8e984f None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:01.915817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420e685-de6b-4ba1-a64a-89ea6c8e984f None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:01.915817 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420e685-de6b-4ba1-a64a-89ea6c8e984f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:01.958685 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-2420e685-de6b-4ba1-a64a-89ea6c8e984f None None] MFA Rules not processed for user `4fe36a87392c40379ac160ebaa3cd84c`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:23:01.963030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2420e685-de6b-4ba1-a64a-89ea6c8e984f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:02.038971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1305/2627] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:23:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:23:02.054271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746fddc-c301-4418-9348-7a5214ce3ddd service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:02.055104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4746fddc-c301-4418-9348-7a5214ce3ddd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:02.108746 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:23:02.135799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746fddc-c301-4418-9348-7a5214ce3ddd service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:02.142566 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65902036-8656-4a82-a2ce-84818554c937 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:02.143196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65902036-8656-4a82-a2ce-84818554c937 service nova] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:02.143478 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65902036-8656-4a82-a2ce-84818554c937 service nova] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:02.143752 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65902036-8656-4a82-a2ce-84818554c937 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:02.152268 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65902036-8656-4a82-a2ce-84818554c937 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:02.153881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65902036-8656-4a82-a2ce-84818554c937 service nova] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:02.164302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1323/2628] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:23:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:23:03.769226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996478e2-2029-4c25-9e44-17fe4369758d None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:03.769808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996478e2-2029-4c25-9e44-17fe4369758d None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:03.769808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996478e2-2029-4c25-9e44-17fe4369758d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:03.790017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-996478e2-2029-4c25-9e44-17fe4369758d None None] MFA Rules not processed for user `4fe36a87392c40379ac160ebaa3cd84c`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:23:03.791240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-996478e2-2029-4c25-9e44-17fe4369758d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:03.798143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1306/2629] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:23:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:23:04.140950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65902036-8656-4a82-a2ce-84818554c937 service nova] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:04.150521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65902036-8656-4a82-a2ce-84818554c937 service nova] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:04.154588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54050a9d-3f2c-410c-bdce-e8e459dc9852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:04.155194 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54050a9d-3f2c-410c-bdce-e8e459dc9852 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:04.155463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54050a9d-3f2c-410c-bdce-e8e459dc9852 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:04.155733 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54050a9d-3f2c-410c-bdce-e8e459dc9852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:04.157767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-54050a9d-3f2c-410c-bdce-e8e459dc9852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:04.167895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54050a9d-3f2c-410c-bdce-e8e459dc9852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:04.169385 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54050a9d-3f2c-410c-bdce-e8e459dc9852 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:04.177776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1324/2630] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:23:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:23:08.966548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b12c1-a1c1-4004-b7ee-d0917e9b680b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:08.967735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b12c1-a1c1-4004-b7ee-d0917e9b680b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:08.968314 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b12c1-a1c1-4004-b7ee-d0917e9b680b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:08.992526 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-665b12c1-a1c1-4004-b7ee-d0917e9b680b None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:23:08.994237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-665b12c1-a1c1-4004-b7ee-d0917e9b680b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:09.064911 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1307/2631] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:23:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:23:09.078308 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54050a9d-3f2c-410c-bdce-e8e459dc9852 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:09.080576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-54050a9d-3f2c-410c-bdce-e8e459dc9852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:09.190428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54050a9d-3f2c-410c-bdce-e8e459dc9852 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:09.206586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ec2fa-5081-4a07-a85d-c54721219c41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:09.207640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ec2fa-5081-4a07-a85d-c54721219c41 service swift] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:09.207803 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ec2fa-5081-4a07-a85d-c54721219c41 service swift] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:09.207985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ec2fa-5081-4a07-a85d-c54721219c41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:09.229141 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ec2fa-5081-4a07-a85d-c54721219c41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:09.230763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ec2fa-5081-4a07-a85d-c54721219c41 service swift] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:09.247482 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1325/2632] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:23:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 01:23:19.052404 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=91 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:23:19.238256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=50 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:23:22.881509 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dbefa-0496-4596-8b6e-d94908df8461 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:22.882093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dbefa-0496-4596-8b6e-d94908df8461 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:22.882093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dbefa-0496-4596-8b6e-d94908df8461 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:22.883320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1308/2633] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:23:22.890562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ce396-cd80-42b2-8d05-5f4c08f2cfe2 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:22.891071 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ce396-cd80-42b2-8d05-5f4c08f2cfe2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:22.891172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ce396-cd80-42b2-8d05-5f4c08f2cfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:22.911008 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6d7ce396-cd80-42b2-8d05-5f4c08f2cfe2 None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:23:22.912332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6d7ce396-cd80-42b2-8d05-5f4c08f2cfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:22.922047 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1326/2634] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:23:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:23:22.933814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dbefa-0496-4596-8b6e-d94908df8461 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:22.935730 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-cc6dbefa-0496-4596-8b6e-d94908df8461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:22.997236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dbefa-0496-4596-8b6e-d94908df8461 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:23.000862 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2718e86-0bb2-402b-9153-95ea75386e0a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c9d444b3294ed18c3d488cf0a3eeb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:23.001464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2718e86-0bb2-402b-9153-95ea75386e0a service glance] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:23.001741 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2718e86-0bb2-402b-9153-95ea75386e0a service glance] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:23.002010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2718e86-0bb2-402b-9153-95ea75386e0a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:23.012658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2718e86-0bb2-402b-9153-95ea75386e0a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:23.014159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2718e86-0bb2-402b-9153-95ea75386e0a service glance] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:23.030919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1309/2635] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:23:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:23:23.099650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1e88b-4e9f-4b38-807b-530d7d0d21d6 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:23.099935 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1e88b-4e9f-4b38-807b-530d7d0d21d6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:23.100242 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1e88b-4e9f-4b38-807b-530d7d0d21d6 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:23.101637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1327/2636] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:23:23.107310 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0ddee-1cd6-4762-9669-f280f60ee69c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:23.107586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0ddee-1cd6-4762-9669-f280f60ee69c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:23.107873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0ddee-1cd6-4762-9669-f280f60ee69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:23.128205 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-a7a0ddee-1cd6-4762-9669-f280f60ee69c None None] MFA Rules not processed for user `86cea2f9238b42e0a7cb26bac494a8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:23:23.129386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a7a0ddee-1cd6-4762-9669-f280f60ee69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:23.135838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1310/2637] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:23:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:23:23.147521 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1e88b-4e9f-4b38-807b-530d7d0d21d6 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:23.159256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1e88b-4e9f-4b38-807b-530d7d0d21d6 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:23.162833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd809ba-1538-485c-beff-eea44258275f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:23.163432 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd809ba-1538-485c-beff-eea44258275f service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:23.163729 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd809ba-1538-485c-beff-eea44258275f service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:23.164019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd809ba-1538-485c-beff-eea44258275f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:23.172769 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd809ba-1538-485c-beff-eea44258275f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:23.174229 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd809ba-1538-485c-beff-eea44258275f service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:23.181926 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1328/2638] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:23:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:23:23.264983 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294c31e-686d-468a-8d84-e523dcc9f367 None None] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:23.265347 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294c31e-686d-468a-8d84-e523dcc9f367 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:23.265625 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294c31e-686d-468a-8d84-e523dcc9f367 None None] PATH_INFO: `/` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:23.266968 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1311/2639] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:23:23.272148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1e114-c7a9-4785-8886-ee4e38e8aca6 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:23.272436 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1e114-c7a9-4785-8886-ee4e38e8aca6 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:23.272709 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1e114-c7a9-4785-8886-ee4e38e8aca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:23.300810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d8a1e114-c7a9-4785-8886-ee4e38e8aca6 None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:23:23.302012 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d8a1e114-c7a9-4785-8886-ee4e38e8aca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:23.312230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1329/2640] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:23:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:23:23.322538 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294c31e-686d-468a-8d84-e523dcc9f367 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:23.334234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294c31e-686d-468a-8d84-e523dcc9f367 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:23.337827 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1d86d-3051-4355-915f-21c2aa3ea5a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:23.338423 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1d86d-3051-4355-915f-21c2aa3ea5a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:23.338695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1d86d-3051-4355-915f-21c2aa3ea5a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:23.338949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1d86d-3051-4355-915f-21c2aa3ea5a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:23.351326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1d86d-3051-4355-915f-21c2aa3ea5a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:23.352947 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1d86d-3051-4355-915f-21c2aa3ea5a7 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:23.367463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1312/2641] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:23:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:23:26.175739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccc402-c543-4492-acae-886c711b9e3b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:26.176220 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccc402-c543-4492-acae-886c711b9e3b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:26.176336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccc402-c543-4492-acae-886c711b9e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:26.196102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-67ccc402-c543-4492-acae-886c711b9e3b None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:23:26.197300 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-67ccc402-c543-4492-acae-886c711b9e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:26.204693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1330/2642] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:23:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:23:26.216514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1d86d-3051-4355-915f-21c2aa3ea5a7 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:26.227529 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1d86d-3051-4355-915f-21c2aa3ea5a7 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:26.233069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f24fd-8bf4-47a5-a637-b01e93fb9ef2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:26.233669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f24fd-8bf4-47a5-a637-b01e93fb9ef2 service swift] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:26.233921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f24fd-8bf4-47a5-a637-b01e93fb9ef2 service swift] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:26.234186 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f24fd-8bf4-47a5-a637-b01e93fb9ef2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:26.242622 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f24fd-8bf4-47a5-a637-b01e93fb9ef2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:26.244176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f24fd-8bf4-47a5-a637-b01e93fb9ef2 service swift] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:26.252978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1313/2643] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:23:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:23:36.190821 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=15 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:23:36.249750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=41 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:23:49.615234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04e4fe-be5c-4b71-9b4a-63e9f810f937 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:49.615703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04e4fe-be5c-4b71-9b4a-63e9f810f937 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:49.615812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04e4fe-be5c-4b71-9b4a-63e9f810f937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:49.635918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-9c04e4fe-be5c-4b71-9b4a-63e9f810f937 None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:23:49.637186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9c04e4fe-be5c-4b71-9b4a-63e9f810f937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:49.644594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1331/2644] 10.222.0.44 () {58 vars in 967 bytes} [Tue Aug 30 01:23:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:23:49.667500 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f24fd-8bf4-47a5-a637-b01e93fb9ef2 service swift] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:49.678663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f24fd-8bf4-47a5-a637-b01e93fb9ef2 service swift] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:49.683624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dea8c4-21af-4306-97f2-17bfcda24e87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:49.684355 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dea8c4-21af-4306-97f2-17bfcda24e87 service swift] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:49.684676 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dea8c4-21af-4306-97f2-17bfcda24e87 service swift] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:49.684942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dea8c4-21af-4306-97f2-17bfcda24e87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:49.694752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dea8c4-21af-4306-97f2-17bfcda24e87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:49.696348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dea8c4-21af-4306-97f2-17bfcda24e87 service swift] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:49.704615 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1314/2645] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:23:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:23:59.629542 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:23:59.701268 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=6 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:24:05.839804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba3c6e-e67f-4ab9-a896-2f073e7e6465 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:05.840527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba3c6e-e67f-4ab9-a896-2f073e7e6465 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:05.840527 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba3c6e-e67f-4ab9-a896-2f073e7e6465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:05.859372 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-4fba3c6e-e67f-4ab9-a896-2f073e7e6465 None None] MFA Rules not processed for user `4fe36a87392c40379ac160ebaa3cd84c`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:24:05.860593 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4fba3c6e-e67f-4ab9-a896-2f073e7e6465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:05.867562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1332/2646] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:24:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:24:15.853601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:24:16.258617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dea8c4-21af-4306-97f2-17bfcda24e87 service swift] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:24:16.260053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-68dea8c4-21af-4306-97f2-17bfcda24e87 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:16.272774 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dea8c4-21af-4306-97f2-17bfcda24e87 service swift] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:24:16.277446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a65e40-4a66-4dba-8f25-6f66f2020a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:24:16.278048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a65e40-4a66-4dba-8f25-6f66f2020a8a service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:16.278324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a65e40-4a66-4dba-8f25-6f66f2020a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:16.278594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a65e40-4a66-4dba-8f25-6f66f2020a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:16.280325 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b6a65e40-4a66-4dba-8f25-6f66f2020a8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:16.341030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a65e40-4a66-4dba-8f25-6f66f2020a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:24:16.342464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a65e40-4a66-4dba-8f25-6f66f2020a8a service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:24:16.351320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1315/2647] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:24:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:24:22.585626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba3c6e-e67f-4ab9-a896-2f073e7e6465 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:24:22.606652 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba3c6e-e67f-4ab9-a896-2f073e7e6465 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:24:22.610596 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e25daf-0a9a-4d42-879b-e43aa4af944a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:24:22.611221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e25daf-0a9a-4d42-879b-e43aa4af944a service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:22.611477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e25daf-0a9a-4d42-879b-e43aa4af944a service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:22.611745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e25daf-0a9a-4d42-879b-e43aa4af944a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:22.613546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-87e25daf-0a9a-4d42-879b-e43aa4af944a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:22.679941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e25daf-0a9a-4d42-879b-e43aa4af944a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:24:22.681409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e25daf-0a9a-4d42-879b-e43aa4af944a service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:24:22.702198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1333/2648] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:24:26.347298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=28 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:24:29.583761 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a65e40-4a66-4dba-8f25-6f66f2020a8a service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:24:29.585425 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b6a65e40-4a66-4dba-8f25-6f66f2020a8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:29.632026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a65e40-4a66-4dba-8f25-6f66f2020a8a service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:24:29.637948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:24:29.638574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:29.638851 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:29.639120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] PATH_INFO: `/v3/users/e76eba6f76f84c5cb006e079aa86157a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:29.640306 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e76eba6f76f84c5cb006e079aa86157a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:24:29.642218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:24:29.702604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e76eba6f76f84c5cb006e079aa86157a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:29.712881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e76eba6f76f84c5cb006e079aa86157a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:29.714281 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e76eba6f76f84c5cb006e079aa86157a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:29.714752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e76eba6f76f84c5cb006e079aa86157a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:29.731123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e76eba6f76f84c5cb006e079aa86157a'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:29.744871 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1316/2649] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:24:29 2022] DELETE /identity/v3/users/e76eba6f76f84c5cb006e079aa86157a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:24:29.926379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e25daf-0a9a-4d42-879b-e43aa4af944a service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:24:29.938254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e25daf-0a9a-4d42-879b-e43aa4af944a service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:24:30.000951 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:24:30.001627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:30.001627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:30.001627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] PATH_INFO: `/v3/projects/089c19009d374769a0d5bd5a82acaaa7` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:30.003410 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] RBAC: Authorizing `identity:delete_project(project_id=089c19009d374769a0d5bd5a82acaaa7)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:24:30.005351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:24:30.065088 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '089c19009d374769a0d5bd5a82acaaa7', 'request_id': 'req-e85d4fb9-092b-49a1-b600-36f5dea4680f'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.076044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '089c19009d374769a0d5bd5a82acaaa7', 'request_id': 'req-e85d4fb9-092b-49a1-b600-36f5dea4680f'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.084234 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '089c19009d374769a0d5bd5a82acaaa7', 'request_id': 'req-e85d4fb9-092b-49a1-b600-36f5dea4680f'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.100364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] The token cache is being invalidate because project 089c19009d374769a0d5bd5a82acaaa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:24:30.100841 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.101859 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.104232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1334/2650] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:24:29 2022] DELETE /identity/v3/projects/089c19009d374769a0d5bd5a82acaaa7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:24:30.108628 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:24:30.110821 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:30.128653 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd952766-f16e-458f-844d-9c2a2a50d789 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:24:30.212920 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975a2fd-6e88-4160-8236-431b4e0d4f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:24:30.213919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975a2fd-6e88-4160-8236-431b4e0d4f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:30.214192 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975a2fd-6e88-4160-8236-431b4e0d4f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:30.214537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975a2fd-6e88-4160-8236-431b4e0d4f39 admin admin] PATH_INFO: `/v3/users/9dd674904a4141208d9cca76a39dfe66` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:30.215738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975a2fd-6e88-4160-8236-431b4e0d4f39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dd674904a4141208d9cca76a39dfe66)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:24:30.217932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975a2fd-6e88-4160-8236-431b4e0d4f39 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:24:30.269998 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-0975a2fd-6e88-4160-8236-431b4e0d4f39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dd674904a4141208d9cca76a39dfe66'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.271848 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:24:30.278816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-0975a2fd-6e88-4160-8236-431b4e0d4f39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dd674904a4141208d9cca76a39dfe66'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.279967 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-0975a2fd-6e88-4160-8236-431b4e0d4f39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dd674904a4141208d9cca76a39dfe66'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.280959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-0975a2fd-6e88-4160-8236-431b4e0d4f39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dd674904a4141208d9cca76a39dfe66'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.294435 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-0975a2fd-6e88-4160-8236-431b4e0d4f39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dd674904a4141208d9cca76a39dfe66'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.309032 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1317/2651] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:24:30 2022] DELETE /identity/v3/users/9dd674904a4141208d9cca76a39dfe66 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:24:30.508751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:24:30.510800 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:30.530621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d4fb9-092b-49a1-b600-36f5dea4680f admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:24:30.536915 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:24:30.596021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:24:30.596695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:30.596972 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:30.597245 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] PATH_INFO: `/v3/projects/c605301a5cd5456bbafb025dc67a0cac` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:30.598371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c605301a5cd5456bbafb025dc67a0cac)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:24:30.600299 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:24:30.647725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c605301a5cd5456bbafb025dc67a0cac', 'request_id': 'req-e00a78d9-07c7-4998-81a6-5de6f4f791e0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.656286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c605301a5cd5456bbafb025dc67a0cac', 'request_id': 'req-e00a78d9-07c7-4998-81a6-5de6f4f791e0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.662061 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c605301a5cd5456bbafb025dc67a0cac', 'request_id': 'req-e00a78d9-07c7-4998-81a6-5de6f4f791e0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.673900 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] The token cache is being invalidate because project c605301a5cd5456bbafb025dc67a0cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:24:30.674183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.674847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:24:30.676375 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1335/2652] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:24:30 2022] DELETE /identity/v3/projects/c605301a5cd5456bbafb025dc67a0cac => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:24:33.124208 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176eea5-5144-4f5b-b6fb-b629fe41893b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:33.124740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176eea5-5144-4f5b-b6fb-b629fe41893b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:33.124740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176eea5-5144-4f5b-b6fb-b629fe41893b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:33.144584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-1176eea5-5144-4f5b-b6fb-b629fe41893b None None] MFA Rules not processed for user `22bf08f4cdb24731bf00c7962bf81c40`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:24:33.145789 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1176eea5-5144-4f5b-b6fb-b629fe41893b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:33.204627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1318/2653] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:24:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:24:40.670816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:24:43.197422 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=37 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:24:49.807016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:24:49.808453 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:49.895694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a78d9-07c7-4998-81a6-5de6f4f791e0 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:24:49.899755 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f10ea3-8cdb-4fa3-a1f6-9332e8e5e4ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:24:49.900973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f10ea3-8cdb-4fa3-a1f6-9332e8e5e4ed service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:49.901252 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f10ea3-8cdb-4fa3-a1f6-9332e8e5e4ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:49.901516 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f10ea3-8cdb-4fa3-a1f6-9332e8e5e4ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:49.903374 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c6f10ea3-8cdb-4fa3-a1f6-9332e8e5e4ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:49.961146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f10ea3-8cdb-4fa3-a1f6-9332e8e5e4ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:24:49.963441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f10ea3-8cdb-4fa3-a1f6-9332e8e5e4ed service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:24:49.972436 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1336/2654] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:24:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:24:56.835531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e9974-0a9f-4e52-924a-8ff3da657dce None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:56.836076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e9974-0a9f-4e52-924a-8ff3da657dce None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:56.836208 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e9974-0a9f-4e52-924a-8ff3da657dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:56.855436 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-790e9974-0a9f-4e52-924a-8ff3da657dce None None] MFA Rules not processed for user `2839103f52324a1ab1ebd43fe230addc`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:24:56.856643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-790e9974-0a9f-4e52-924a-8ff3da657dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:56.863824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1319/2655] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:24:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:24:58.199772 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f10ea3-8cdb-4fa3-a1f6-9332e8e5e4ed service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:24:58.201280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c6f10ea3-8cdb-4fa3-a1f6-9332e8e5e4ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:58.295737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f10ea3-8cdb-4fa3-a1f6-9332e8e5e4ed service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:24:58.300880 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37babd-7676-435a-8740-e43607044cb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08f34295f19b4d6ea405f1e8c8190db5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:24:58.301490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37babd-7676-435a-8740-e43607044cb0 service placement] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:58.301767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37babd-7676-435a-8740-e43607044cb0 service placement] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:58.302034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37babd-7676-435a-8740-e43607044cb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:58.303785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-1f37babd-7676-435a-8740-e43607044cb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:58.315426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:24:58.317986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37babd-7676-435a-8740-e43607044cb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:24:58.320120 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37babd-7676-435a-8740-e43607044cb0 service placement] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:24:58.332236 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1337/2656] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:24:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:25:06.850039 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:25:08.327818 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:25:10.202384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8656d-a7c0-4f70-bfc4-eb507b329e0b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:25:10.202901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8656d-a7c0-4f70-bfc4-eb507b329e0b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:25:10.202901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8656d-a7c0-4f70-bfc4-eb507b329e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:25:10.222994 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-90d8656d-a7c0-4f70-bfc4-eb507b329e0b None None] MFA Rules not processed for user `4fe36a87392c40379ac160ebaa3cd84c`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:25:10.224263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-90d8656d-a7c0-4f70-bfc4-eb507b329e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:25:10.282564 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1320/2657] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:25:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:25:20.275859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=25 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:25:43.335063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c16ce-467c-4ce2-b88d-27421e3c804c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:25:43.336205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c16ce-467c-4ce2-b88d-27421e3c804c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:25:43.336205 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c16ce-467c-4ce2-b88d-27421e3c804c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:25:43.371451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-5d5c16ce-467c-4ce2-b88d-27421e3c804c None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:25:43.372706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-5d5c16ce-467c-4ce2-b88d-27421e3c804c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:25:43.443887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1338/2658] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:25:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:25:43.492344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8656d-a7c0-4f70-bfc4-eb507b329e0b None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:25:43.493750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-90d8656d-a7c0-4f70-bfc4-eb507b329e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:25:43.560960 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8656d-a7c0-4f70-bfc4-eb507b329e0b None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:25:43.565898 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec45b2f-9319-4d06-aed3-da8a76fcc158 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:25:43.566496 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec45b2f-9319-4d06-aed3-da8a76fcc158 service swift] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:25:43.566766 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec45b2f-9319-4d06-aed3-da8a76fcc158 service swift] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:25:43.567734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec45b2f-9319-4d06-aed3-da8a76fcc158 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:25:43.575880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec45b2f-9319-4d06-aed3-da8a76fcc158 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:25:43.577401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec45b2f-9319-4d06-aed3-da8a76fcc158 service swift] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:25:43.585224 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1321/2659] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:25:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:25:53.433233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=32 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:25:53.582553 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=32 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:25:56.559124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f18697-1697-4adb-b24d-d815eefd5374 None None] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:25:56.559691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f18697-1697-4adb-b24d-d815eefd5374 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:25:56.559691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f18697-1697-4adb-b24d-d815eefd5374 None None] PATH_INFO: `/` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:25:56.561133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1339/2660] 10.222.0.44 () {54 vars in 921 bytes} [Tue Aug 30 01:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:25:56.567159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478edbee-d6ca-411a-8717-16c4dc309eba None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:25:56.567710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478edbee-d6ca-411a-8717-16c4dc309eba None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:25:56.567809 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478edbee-d6ca-411a-8717-16c4dc309eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:25:56.617464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-478edbee-d6ca-411a-8717-16c4dc309eba None None] MFA Rules not processed for user `049399a752ed4646a13f8e4a3a62d8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:25:56.618755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-478edbee-d6ca-411a-8717-16c4dc309eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:25:56.699589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1322/2661] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Aug 30 01:25:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:25:56.711191 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f18697-1697-4adb-b24d-d815eefd5374 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:25:56.713119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-10f18697-1697-4adb-b24d-d815eefd5374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:25:56.775497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f18697-1697-4adb-b24d-d815eefd5374 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:25:56.779768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa77c6-9089-4670-89c4-27bef91bd85d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c9d444b3294ed18c3d488cf0a3eeb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:25:56.781215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa77c6-9089-4670-89c4-27bef91bd85d service glance] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:25:56.781487 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa77c6-9089-4670-89c4-27bef91bd85d service glance] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:25:56.781754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa77c6-9089-4670-89c4-27bef91bd85d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:25:56.793030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa77c6-9089-4670-89c4-27bef91bd85d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:25:56.794441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa77c6-9089-4670-89c4-27bef91bd85d service glance] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:25:56.806000 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1340/2662] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:25:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:25:58.736123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6769ad-3677-4211-a263-f9d60af2d599 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:25:58.736575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6769ad-3677-4211-a263-f9d60af2d599 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:25:58.736714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6769ad-3677-4211-a263-f9d60af2d599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:25:58.756858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-2b6769ad-3677-4211-a263-f9d60af2d599 None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:25:58.758057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2b6769ad-3677-4211-a263-f9d60af2d599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:25:58.765144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1323/2663] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:25:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:25:58.777988 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa77c6-9089-4670-89c4-27bef91bd85d service glance] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:25:58.790741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa77c6-9089-4670-89c4-27bef91bd85d service glance] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:25:58.795882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429ecea-5360-47b1-b152-24c885af329a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:25:58.796518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429ecea-5360-47b1-b152-24c885af329a service swift] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:25:58.796796 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429ecea-5360-47b1-b152-24c885af329a service swift] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:25:58.797066 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429ecea-5360-47b1-b152-24c885af329a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:25:58.806028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429ecea-5360-47b1-b152-24c885af329a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:25:58.807669 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429ecea-5360-47b1-b152-24c885af329a service swift] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:25:58.816230 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1341/2664] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:25:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:26:08.750864 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=47 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:26:08.812627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=32 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:26:17.759447 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93656d27-1e14-4c36-9261-1f0c1f21cfb5 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:26:17.759986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93656d27-1e14-4c36-9261-1f0c1f21cfb5 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:26:17.759986 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93656d27-1e14-4c36-9261-1f0c1f21cfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:26:17.779582 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-93656d27-1e14-4c36-9261-1f0c1f21cfb5 None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:26:17.780807 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-93656d27-1e14-4c36-9261-1f0c1f21cfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:26:17.788236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1324/2665] 10.222.0.44 () {58 vars in 967 bytes} [Tue Aug 30 01:26:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:26:17.800631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429ecea-5360-47b1-b152-24c885af329a service swift] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:26:17.811672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429ecea-5360-47b1-b152-24c885af329a service swift] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:26:17.817511 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae136069-3f21-488d-a1af-bd8f5372f870 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:26:17.818144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae136069-3f21-488d-a1af-bd8f5372f870 service swift] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:26:17.818423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae136069-3f21-488d-a1af-bd8f5372f870 service swift] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:26:17.818695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae136069-3f21-488d-a1af-bd8f5372f870 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:26:17.827030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae136069-3f21-488d-a1af-bd8f5372f870 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:26:17.828651 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae136069-3f21-488d-a1af-bd8f5372f870 service swift] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:26:17.837365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1342/2666] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:26:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:26:26.672621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf15570-230f-48a6-92d1-5749a0cacde1 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:26:26.673133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf15570-230f-48a6-92d1-5749a0cacde1 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:26:26.673133 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf15570-230f-48a6-92d1-5749a0cacde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:26:26.692389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-daf15570-230f-48a6-92d1-5749a0cacde1 None None] MFA Rules not processed for user `4fe36a87392c40379ac160ebaa3cd84c`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:26:26.693606 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-daf15570-230f-48a6-92d1-5749a0cacde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:26:26.701472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1325/2667] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:26:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:26:26.938724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae136069-3f21-488d-a1af-bd8f5372f870 service swift] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:26:26.940212 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ae136069-3f21-488d-a1af-bd8f5372f870 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:26:27.018092 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae136069-3f21-488d-a1af-bd8f5372f870 service swift] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:26:27.021715 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf911e-49c6-43cf-b257-dc5a10d2293e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:26:27.022292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf911e-49c6-43cf-b257-dc5a10d2293e service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:26:27.022562 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf911e-49c6-43cf-b257-dc5a10d2293e service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:26:27.022827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf911e-49c6-43cf-b257-dc5a10d2293e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:26:27.024746 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3bdf911e-49c6-43cf-b257-dc5a10d2293e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:26:27.037813 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf911e-49c6-43cf-b257-dc5a10d2293e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:26:27.039222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf911e-49c6-43cf-b257-dc5a10d2293e service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:26:27.050246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1343/2668] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:26:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:26:29.694901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf15570-230f-48a6-92d1-5749a0cacde1 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:26:29.696964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-daf15570-230f-48a6-92d1-5749a0cacde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:26:29.713447 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf15570-230f-48a6-92d1-5749a0cacde1 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:26:29.719434 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79162fb3-07ff-48c0-9eda-1545cda03ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:26:29.720288 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79162fb3-07ff-48c0-9eda-1545cda03ced service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:26:29.720698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79162fb3-07ff-48c0-9eda-1545cda03ced service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:26:29.721068 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79162fb3-07ff-48c0-9eda-1545cda03ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:26:29.722900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-79162fb3-07ff-48c0-9eda-1545cda03ced service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:26:29.737233 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79162fb3-07ff-48c0-9eda-1545cda03ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:26:29.738661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79162fb3-07ff-48c0-9eda-1545cda03ced service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:26:29.749780 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1326/2669] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:26:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:26:37.045222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=41 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:26:39.744870 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=12 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:26:59.650034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf911e-49c6-43cf-b257-dc5a10d2293e service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:26:59.659977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf911e-49c6-43cf-b257-dc5a10d2293e service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:26:59.663684 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d53db1-6824-4a0b-89e1-0d14fa8e0781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:26:59.664317 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d53db1-6824-4a0b-89e1-0d14fa8e0781 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:26:59.664598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d53db1-6824-4a0b-89e1-0d14fa8e0781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:26:59.664873 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d53db1-6824-4a0b-89e1-0d14fa8e0781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:26:59.666741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-29d53db1-6824-4a0b-89e1-0d14fa8e0781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:26:59.731793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d53db1-6824-4a0b-89e1-0d14fa8e0781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:26:59.733295 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d53db1-6824-4a0b-89e1-0d14fa8e0781 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:26:59.743759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1344/2670] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:26:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:03.330522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79162fb3-07ff-48c0-9eda-1545cda03ced service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:03.344070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79162fb3-07ff-48c0-9eda-1545cda03ced service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:03.350565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07635bc6-71db-428a-8e26-1b6e26153f38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08f34295f19b4d6ea405f1e8c8190db5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:03.351371 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07635bc6-71db-428a-8e26-1b6e26153f38 service placement] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:03.351696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07635bc6-71db-428a-8e26-1b6e26153f38 service placement] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:03.352103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07635bc6-71db-428a-8e26-1b6e26153f38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:03.354221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-07635bc6-71db-428a-8e26-1b6e26153f38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:03.376747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07635bc6-71db-428a-8e26-1b6e26153f38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:03.378542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07635bc6-71db-428a-8e26-1b6e26153f38 service placement] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:03.392226 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1327/2671] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:03.412955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d53db1-6824-4a0b-89e1-0d14fa8e0781 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:03.415636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-29d53db1-6824-4a0b-89e1-0d14fa8e0781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:03.432665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d53db1-6824-4a0b-89e1-0d14fa8e0781 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:03.438305 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee043e-5a83-466d-bceb-a2a8739564e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:03.439342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee043e-5a83-466d-bceb-a2a8739564e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:03.439691 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee043e-5a83-466d-bceb-a2a8739564e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:03.440090 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee043e-5a83-466d-bceb-a2a8739564e4 admin admin] PATH_INFO: `/v3/users/22bf08f4cdb24731bf00c7962bf81c40` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:03.442438 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee043e-5a83-466d-bceb-a2a8739564e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22bf08f4cdb24731bf00c7962bf81c40)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:03.444977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee043e-5a83-466d-bceb-a2a8739564e4 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:03.510668 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4bee043e-5a83-466d-bceb-a2a8739564e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22bf08f4cdb24731bf00c7962bf81c40'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:03.525425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4bee043e-5a83-466d-bceb-a2a8739564e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22bf08f4cdb24731bf00c7962bf81c40'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:03.532629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4bee043e-5a83-466d-bceb-a2a8739564e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22bf08f4cdb24731bf00c7962bf81c40'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:03.541112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4bee043e-5a83-466d-bceb-a2a8739564e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22bf08f4cdb24731bf00c7962bf81c40'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:03.541736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-4bee043e-5a83-466d-bceb-a2a8739564e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22bf08f4cdb24731bf00c7962bf81c40'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:03.547979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1345/2672] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:03 2022] DELETE /identity/v3/users/22bf08f4cdb24731bf00c7962bf81c40 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:03.749706 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07635bc6-71db-428a-8e26-1b6e26153f38 service placement] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:03.766832 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07635bc6-71db-428a-8e26-1b6e26153f38 service placement] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:03.840728 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a68193-441e-4a4d-90ff-cb8beae1c163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:03.841476 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a68193-441e-4a4d-90ff-cb8beae1c163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:03.841824 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a68193-441e-4a4d-90ff-cb8beae1c163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:03.842109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a68193-441e-4a4d-90ff-cb8beae1c163 admin admin] PATH_INFO: `/v3/projects/5d08b735ca4249309f4ecd40cbacd537` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:03.843266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a68193-441e-4a4d-90ff-cb8beae1c163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d08b735ca4249309f4ecd40cbacd537)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:03.845244 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a68193-441e-4a4d-90ff-cb8beae1c163 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:03.896921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-33a68193-441e-4a4d-90ff-cb8beae1c163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d08b735ca4249309f4ecd40cbacd537', 'request_id': 'req-33a68193-441e-4a4d-90ff-cb8beae1c163'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:03.903070 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-33a68193-441e-4a4d-90ff-cb8beae1c163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d08b735ca4249309f4ecd40cbacd537', 'request_id': 'req-33a68193-441e-4a4d-90ff-cb8beae1c163'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:03.908928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-33a68193-441e-4a4d-90ff-cb8beae1c163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d08b735ca4249309f4ecd40cbacd537', 'request_id': 'req-33a68193-441e-4a4d-90ff-cb8beae1c163'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:03.922420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-33a68193-441e-4a4d-90ff-cb8beae1c163 admin admin] The token cache is being invalidate because project 5d08b735ca4249309f4ecd40cbacd537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:03.922720 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-33a68193-441e-4a4d-90ff-cb8beae1c163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:03.923436 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-33a68193-441e-4a4d-90ff-cb8beae1c163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:03.925025 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1328/2673] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:03 2022] DELETE /identity/v3/projects/5d08b735ca4249309f4ecd40cbacd537 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:03.930335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d63a8-3cde-4964-a7a4-ff80a31bf0d9 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:03.930630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d63a8-3cde-4964-a7a4-ff80a31bf0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:03.930907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d63a8-3cde-4964-a7a4-ff80a31bf0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:03.979507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-db0d63a8-3cde-4964-a7a4-ff80a31bf0d9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:03.980762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-db0d63a8-3cde-4964-a7a4-ff80a31bf0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:04.021745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:04.043415 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1346/2674] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:04.051172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38409d45-5bb8-4b91-86b0-19b89264ab37 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:04.051455 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38409d45-5bb8-4b91-86b0-19b89264ab37 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:04.051735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38409d45-5bb8-4b91-86b0-19b89264ab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:04.072545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-38409d45-5bb8-4b91-86b0-19b89264ab37 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:04.073765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-38409d45-5bb8-4b91-86b0-19b89264ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:04.082549 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1329/2675] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:04.086886 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d63a8-3cde-4964-a7a4-ff80a31bf0d9 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:04.096644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d63a8-3cde-4964-a7a4-ff80a31bf0d9 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:04.101957 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62b5eb-d6f7-49fe-8b27-38397efbfc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:04.102570 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62b5eb-d6f7-49fe-8b27-38397efbfc66 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:04.102845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62b5eb-d6f7-49fe-8b27-38397efbfc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:04.103112 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62b5eb-d6f7-49fe-8b27-38397efbfc66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:04.103757 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62b5eb-d6f7-49fe-8b27-38397efbfc66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:27:04.104449 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62b5eb-d6f7-49fe-8b27-38397efbfc66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:04.106472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62b5eb-d6f7-49fe-8b27-38397efbfc66 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:04.118805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1347/2676] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:27:04.123178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38409d45-5bb8-4b91-86b0-19b89264ab37 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:04.133052 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38409d45-5bb8-4b91-86b0-19b89264ab37 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:04.137687 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab690785-0d2a-42ae-8f61-3c69b18484d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:04.138276 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690785-0d2a-42ae-8f61-3c69b18484d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:04.138557 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690785-0d2a-42ae-8f61-3c69b18484d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:04.138823 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690785-0d2a-42ae-8f61-3c69b18484d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:04.139466 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab690785-0d2a-42ae-8f61-3c69b18484d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:04.141376 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab690785-0d2a-42ae-8f61-3c69b18484d4 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:04.179907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1330/2677] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:04 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:04.183831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62b5eb-d6f7-49fe-8b27-38397efbfc66 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:04.195960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62b5eb-d6f7-49fe-8b27-38397efbfc66 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:04.258774 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94b934-0b04-416b-8e6f-50b9d9067b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:04.259407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94b934-0b04-416b-8e6f-50b9d9067b09 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:04.259690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94b934-0b04-416b-8e6f-50b9d9067b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:04.259963 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94b934-0b04-416b-8e6f-50b9d9067b09 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:04.260654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94b934-0b04-416b-8e6f-50b9d9067b09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:04.262614 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94b934-0b04-416b-8e6f-50b9d9067b09 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:04.289464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1348/2678] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:04 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:04.290961 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab690785-0d2a-42ae-8f61-3c69b18484d4 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:04.302035 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab690785-0d2a-42ae-8f61-3c69b18484d4 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:04.307083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894f9d9-a8a8-4075-9105-dcf037fefded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:04.307749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894f9d9-a8a8-4075-9105-dcf037fefded admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:04.308111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894f9d9-a8a8-4075-9105-dcf037fefded admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:04.308439 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894f9d9-a8a8-4075-9105-dcf037fefded admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:04.309059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894f9d9-a8a8-4075-9105-dcf037fefded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:04.310723 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894f9d9-a8a8-4075-9105-dcf037fefded admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:04.322020 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1331/2679] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:04.326026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94b934-0b04-416b-8e6f-50b9d9067b09 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:04.337954 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94b934-0b04-416b-8e6f-50b9d9067b09 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:04.342665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72688610-1ba6-4309-aa72-8d7783280952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:04.343288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72688610-1ba6-4309-aa72-8d7783280952 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:04.343568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72688610-1ba6-4309-aa72-8d7783280952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:04.343869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72688610-1ba6-4309-aa72-8d7783280952 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:04.344531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72688610-1ba6-4309-aa72-8d7783280952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:04.346130 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72688610-1ba6-4309-aa72-8d7783280952 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:04.357203 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1349/2680] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:04.361438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894f9d9-a8a8-4075-9105-dcf037fefded admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:04.372948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894f9d9-a8a8-4075-9105-dcf037fefded admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:04.377795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f270e-1b28-45a5-a7c4-efbb79bb2ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:04.379095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f270e-1b28-45a5-a7c4-efbb79bb2ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:04.379370 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f270e-1b28-45a5-a7c4-efbb79bb2ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:04.379631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f270e-1b28-45a5-a7c4-efbb79bb2ed7 admin admin] PATH_INFO: `/v3/projects/2327fae7aef3473c8ff5d88a7c0c6ab9/users/533dbf811cbf4c8eaea5213bf135a071/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:04.396252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f270e-1b28-45a5-a7c4-efbb79bb2ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2327fae7aef3473c8ff5d88a7c0c6ab9, user_id=533dbf811cbf4c8eaea5213bf135a071, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:04.399131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f270e-1b28-45a5-a7c4-efbb79bb2ed7 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:04.415816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1332/2681] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:04 2022] PUT /identity/v3/projects/2327fae7aef3473c8ff5d88a7c0c6ab9/users/533dbf811cbf4c8eaea5213bf135a071/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:04.427571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437dabc-dff7-4927-b659-cc8de67174f7 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:04.428107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437dabc-dff7-4927-b659-cc8de67174f7 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:04.428221 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437dabc-dff7-4927-b659-cc8de67174f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:04.460419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-9437dabc-dff7-4927-b659-cc8de67174f7 None None] MFA Rules not processed for user `533dbf811cbf4c8eaea5213bf135a071`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:04.461630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9437dabc-dff7-4927-b659-cc8de67174f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:04.529759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1350/2682] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:04.541611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f270e-1b28-45a5-a7c4-efbb79bb2ed7 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:04.543335 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e90f270e-1b28-45a5-a7c4-efbb79bb2ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:04.549826 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:04.629106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f270e-1b28-45a5-a7c4-efbb79bb2ed7 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:04.634661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3d304-17d7-40c7-a2ad-c477d74ba179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:04.635313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3d304-17d7-40c7-a2ad-c477d74ba179 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:04.635594 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3d304-17d7-40c7-a2ad-c477d74ba179 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:04.635875 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3d304-17d7-40c7-a2ad-c477d74ba179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:04.644801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3d304-17d7-40c7-a2ad-c477d74ba179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:04.646374 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3d304-17d7-40c7-a2ad-c477d74ba179 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:04.658397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1333/2683] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:07.375749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437dabc-dff7-4927-b659-cc8de67174f7 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:07.421128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:07.443316 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437dabc-dff7-4927-b659-cc8de67174f7 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:07.448262 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:07.448914 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:07.449186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:07.449451 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] PATH_INFO: `/v3/users/533dbf811cbf4c8eaea5213bf135a071` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:07.450599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=533dbf811cbf4c8eaea5213bf135a071)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:07.452550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:07.502742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '533dbf811cbf4c8eaea5213bf135a071'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.516022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '533dbf811cbf4c8eaea5213bf135a071'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.524219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '533dbf811cbf4c8eaea5213bf135a071'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.532206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '533dbf811cbf4c8eaea5213bf135a071'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.532808 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '533dbf811cbf4c8eaea5213bf135a071'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.539595 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1351/2684] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:07 2022] DELETE /identity/v3/users/533dbf811cbf4c8eaea5213bf135a071 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:07.544232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9ce47-606a-438c-bb9b-09b81d6dae27 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:07.544715 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9ce47-606a-438c-bb9b-09b81d6dae27 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:07.544813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9ce47-606a-438c-bb9b-09b81d6dae27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:07.564548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-79a9ce47-606a-438c-bb9b-09b81d6dae27 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:07.566057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-79a9ce47-606a-438c-bb9b-09b81d6dae27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:07.630719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1334/2685] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:07.669998 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:07.671964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:07.735702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e55e8-0f4f-435c-a4d1-77bab746f281 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:07.739938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dadd2c-3d55-43ff-87cc-58be58e23204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:07.740890 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dadd2c-3d55-43ff-87cc-58be58e23204 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:07.740890 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dadd2c-3d55-43ff-87cc-58be58e23204 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:07.741513 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dadd2c-3d55-43ff-87cc-58be58e23204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:07.752307 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dadd2c-3d55-43ff-87cc-58be58e23204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:07.753887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dadd2c-3d55-43ff-87cc-58be58e23204 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:07.765376 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1352/2686] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:07.924099 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9ce47-606a-438c-bb9b-09b81d6dae27 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:07.953883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9ce47-606a-438c-bb9b-09b81d6dae27 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:07.969663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a782da-81c2-4a5d-a4ae-a7d85607880c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:07.973787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a782da-81c2-4a5d-a4ae-a7d85607880c service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:07.973787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a782da-81c2-4a5d-a4ae-a7d85607880c service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:07.973787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a782da-81c2-4a5d-a4ae-a7d85607880c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:07.973787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-06a782da-81c2-4a5d-a4ae-a7d85607880c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:07.997023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a782da-81c2-4a5d-a4ae-a7d85607880c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:07.998530 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a782da-81c2-4a5d-a4ae-a7d85607880c service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:08.009614 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1335/2687] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:27:08.167719 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dadd2c-3d55-43ff-87cc-58be58e23204 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:08.169239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-60dadd2c-3d55-43ff-87cc-58be58e23204 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:08.183303 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dadd2c-3d55-43ff-87cc-58be58e23204 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:08.190546 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:08.191250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:08.191598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:08.193683 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf admin admin] PATH_INFO: `/v3/projects/2327fae7aef3473c8ff5d88a7c0c6ab9` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:08.197665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2327fae7aef3473c8ff5d88a7c0c6ab9)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:08.199627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:08.267559 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2327fae7aef3473c8ff5d88a7c0c6ab9', 'request_id': 'req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:08.276633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2327fae7aef3473c8ff5d88a7c0c6ab9', 'request_id': 'req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:08.278032 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:08.284390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2327fae7aef3473c8ff5d88a7c0c6ab9', 'request_id': 'req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:08.297815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf admin admin] The token cache is being invalidate because project 2327fae7aef3473c8ff5d88a7c0c6ab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:08.298110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:08.298780 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-91ad2ad3-62f8-4eb2-a76e-6de3989b4caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:08.300334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1353/2688] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:08 2022] DELETE /identity/v3/projects/2327fae7aef3473c8ff5d88a7c0c6ab9 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:08.305867 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453be32-dacc-4ab0-b374-0e35555cbeb3 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:08.306146 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453be32-dacc-4ab0-b374-0e35555cbeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:08.306412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453be32-dacc-4ab0-b374-0e35555cbeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:08.326479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-7453be32-dacc-4ab0-b374-0e35555cbeb3 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:08.327683 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7453be32-dacc-4ab0-b374-0e35555cbeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:08.404160 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1336/2689] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:08.407625 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e25d0f-15ab-4919-8863-6d49feff65fd None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:08.408181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e25d0f-15ab-4919-8863-6d49feff65fd None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:08.408275 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e25d0f-15ab-4919-8863-6d49feff65fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:08.429973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b3e25d0f-15ab-4919-8863-6d49feff65fd None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:08.431199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b3e25d0f-15ab-4919-8863-6d49feff65fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:08.439488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1354/2690] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:08.444591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453be32-dacc-4ab0-b374-0e35555cbeb3 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:08.454395 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453be32-dacc-4ab0-b374-0e35555cbeb3 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:08.460131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2f667-ec05-4f3a-86db-8e3a85f3398c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:08.460733 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2f667-ec05-4f3a-86db-8e3a85f3398c admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:08.461012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2f667-ec05-4f3a-86db-8e3a85f3398c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:08.461275 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2f667-ec05-4f3a-86db-8e3a85f3398c admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:08.461924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2f667-ec05-4f3a-86db-8e3a85f3398c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:27:08.462225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2f667-ec05-4f3a-86db-8e3a85f3398c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:08.463812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2f667-ec05-4f3a-86db-8e3a85f3398c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:08.477261 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1337/2691] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:27:08.483027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e25d0f-15ab-4919-8863-6d49feff65fd None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:08.493656 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e25d0f-15ab-4919-8863-6d49feff65fd None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:08.498445 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eeee8-77a2-49f8-b4e2-448c231b9269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:08.499068 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eeee8-77a2-49f8-b4e2-448c231b9269 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:08.499344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eeee8-77a2-49f8-b4e2-448c231b9269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:08.499613 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eeee8-77a2-49f8-b4e2-448c231b9269 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:08.500293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9eeee8-77a2-49f8-b4e2-448c231b9269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:08.502190 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9eeee8-77a2-49f8-b4e2-448c231b9269 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:08.537510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1355/2692] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:08 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:08.541669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2f667-ec05-4f3a-86db-8e3a85f3398c admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:08.548584 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:08.553209 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2f667-ec05-4f3a-86db-8e3a85f3398c admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:08.612160 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684b220-c5e4-4366-afbc-02079d547ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:08.612767 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684b220-c5e4-4366-afbc-02079d547ccd admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:08.613036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684b220-c5e4-4366-afbc-02079d547ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:08.613356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684b220-c5e4-4366-afbc-02079d547ccd admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:08.614121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684b220-c5e4-4366-afbc-02079d547ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:08.616670 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684b220-c5e4-4366-afbc-02079d547ccd admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:08.641014 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1338/2693] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:08 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:08.645209 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eeee8-77a2-49f8-b4e2-448c231b9269 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:08.657296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eeee8-77a2-49f8-b4e2-448c231b9269 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:08.662971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edea069-c918-4798-9a74-251f4e0a782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:08.663606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edea069-c918-4798-9a74-251f4e0a782e admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:08.663907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edea069-c918-4798-9a74-251f4e0a782e admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:08.664216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edea069-c918-4798-9a74-251f4e0a782e admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:08.664871 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edea069-c918-4798-9a74-251f4e0a782e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:08.666544 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edea069-c918-4798-9a74-251f4e0a782e admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:08.678906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1356/2694] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:27:08.683312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684b220-c5e4-4366-afbc-02079d547ccd admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:08.694459 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684b220-c5e4-4366-afbc-02079d547ccd admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:08.699154 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3e833-90a3-4179-a848-cf3dc1425d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:08.699759 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3e833-90a3-4179-a848-cf3dc1425d7e admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:08.700069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3e833-90a3-4179-a848-cf3dc1425d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:08.700343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3e833-90a3-4179-a848-cf3dc1425d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:08.700988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3e833-90a3-4179-a848-cf3dc1425d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:08.703180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3e833-90a3-4179-a848-cf3dc1425d7e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:08.717935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1339/2695] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:27:08.725167 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edea069-c918-4798-9a74-251f4e0a782e admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:08.735851 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edea069-c918-4798-9a74-251f4e0a782e admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:08.740826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d396645-e373-42f0-9dea-0e26c5863919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:08.741771 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d396645-e373-42f0-9dea-0e26c5863919 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:08.742054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d396645-e373-42f0-9dea-0e26c5863919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:08.742330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d396645-e373-42f0-9dea-0e26c5863919 admin admin] PATH_INFO: `/v3/projects/50e8cb7056d64587a9e3fff212a1373f/users/038ff31455104b9e904cf9a3746efdfe/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:08.759457 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d396645-e373-42f0-9dea-0e26c5863919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e8cb7056d64587a9e3fff212a1373f, user_id=038ff31455104b9e904cf9a3746efdfe, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:08.762987 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d396645-e373-42f0-9dea-0e26c5863919 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:08.777031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1357/2696] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:08 2022] PUT /identity/v3/projects/50e8cb7056d64587a9e3fff212a1373f/users/038ff31455104b9e904cf9a3746efdfe/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:08.787382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917571a-89e3-49a7-a961-4de7a4e32ea7 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:08.787657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917571a-89e3-49a7-a961-4de7a4e32ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:08.787924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917571a-89e3-49a7-a961-4de7a4e32ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:08.816973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-7917571a-89e3-49a7-a961-4de7a4e32ea7 None None] MFA Rules not processed for user `038ff31455104b9e904cf9a3746efdfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:08.818798 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-7917571a-89e3-49a7-a961-4de7a4e32ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:08.883589 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1340/2697] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:08.888147 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d396645-e373-42f0-9dea-0e26c5863919 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:08.950472 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d396645-e373-42f0-9dea-0e26c5863919 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:08.955356 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232880b-45ca-403a-9ec0-d1523db6b8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:08.956046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232880b-45ca-403a-9ec0-d1523db6b8ab admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:08.956322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232880b-45ca-403a-9ec0-d1523db6b8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:08.956591 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232880b-45ca-403a-9ec0-d1523db6b8ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:08.957231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232880b-45ca-403a-9ec0-d1523db6b8ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:08.959124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232880b-45ca-403a-9ec0-d1523db6b8ab admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:08.986475 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1358/2698] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:08 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:08.991179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917571a-89e3-49a7-a961-4de7a4e32ea7 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:09.001874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917571a-89e3-49a7-a961-4de7a4e32ea7 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:09.059273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a355b27-8bf6-432c-bfc2-1eb39f375716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:09.059881 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a355b27-8bf6-432c-bfc2-1eb39f375716 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:09.060179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a355b27-8bf6-432c-bfc2-1eb39f375716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:09.060446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a355b27-8bf6-432c-bfc2-1eb39f375716 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:09.061067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a355b27-8bf6-432c-bfc2-1eb39f375716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:09.062958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a355b27-8bf6-432c-bfc2-1eb39f375716 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:09.086973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1341/2699] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:08 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:09.091046 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232880b-45ca-403a-9ec0-d1523db6b8ab admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:09.101998 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232880b-45ca-403a-9ec0-d1523db6b8ab admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:09.107355 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b6660-21c7-499d-a1cc-c6e0223a0729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:09.108104 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b6660-21c7-499d-a1cc-c6e0223a0729 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:09.108392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b6660-21c7-499d-a1cc-c6e0223a0729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:09.108664 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b6660-21c7-499d-a1cc-c6e0223a0729 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:09.110589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b6660-21c7-499d-a1cc-c6e0223a0729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:09.112365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b6660-21c7-499d-a1cc-c6e0223a0729 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:09.124494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1359/2700] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:09.128939 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a355b27-8bf6-432c-bfc2-1eb39f375716 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:09.135055 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:09.140975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a355b27-8bf6-432c-bfc2-1eb39f375716 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:09.146518 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b5890-8b8c-4763-a042-721532e78050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:09.147210 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b5890-8b8c-4763-a042-721532e78050 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:09.147488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b5890-8b8c-4763-a042-721532e78050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:09.147762 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b5890-8b8c-4763-a042-721532e78050 admin admin] PATH_INFO: `/v3/projects/5f0f1ef0450e4f0bbc04d5004e68d917/users/ef99e1e6bf5f4b7cbd9c702bb479d8ea/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:09.166597 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b5890-8b8c-4763-a042-721532e78050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0f1ef0450e4f0bbc04d5004e68d917, user_id=ef99e1e6bf5f4b7cbd9c702bb479d8ea, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:09.169323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b5890-8b8c-4763-a042-721532e78050 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:09.184578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1342/2701] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:09 2022] PUT /identity/v3/projects/5f0f1ef0450e4f0bbc04d5004e68d917/users/ef99e1e6bf5f4b7cbd9c702bb479d8ea/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:09.189025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b6660-21c7-499d-a1cc-c6e0223a0729 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:09.201326 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b6660-21c7-499d-a1cc-c6e0223a0729 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:09.255041 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:09.262283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2980c7-b9f3-4b65-990c-039842209a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:09.262902 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2980c7-b9f3-4b65-990c-039842209a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:09.263444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2980c7-b9f3-4b65-990c-039842209a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:09.263973 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2980c7-b9f3-4b65-990c-039842209a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:09.265197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2980c7-b9f3-4b65-990c-039842209a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:09.266898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2980c7-b9f3-4b65-990c-039842209a9a admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:09.278786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1360/2702] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:09.283145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b5890-8b8c-4763-a042-721532e78050 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:09.294134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b5890-8b8c-4763-a042-721532e78050 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:09.298901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94316bd7-c3d4-4d18-813b-cb300925242e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:09.299562 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94316bd7-c3d4-4d18-813b-cb300925242e admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:09.299834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94316bd7-c3d4-4d18-813b-cb300925242e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:09.300212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94316bd7-c3d4-4d18-813b-cb300925242e admin admin] PATH_INFO: `/v3/projects/5f0f1ef0450e4f0bbc04d5004e68d917/users/ef99e1e6bf5f4b7cbd9c702bb479d8ea/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:09.303185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94316bd7-c3d4-4d18-813b-cb300925242e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0f1ef0450e4f0bbc04d5004e68d917, user_id=ef99e1e6bf5f4b7cbd9c702bb479d8ea, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:09.306097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94316bd7-c3d4-4d18-813b-cb300925242e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:09.317348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1343/2703] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:09 2022] PUT /identity/v3/projects/5f0f1ef0450e4f0bbc04d5004e68d917/users/ef99e1e6bf5f4b7cbd9c702bb479d8ea/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:09.328371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bf2c1-cb69-4e59-b03c-68543810cf62 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:09.328658 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bf2c1-cb69-4e59-b03c-68543810cf62 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:09.328934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bf2c1-cb69-4e59-b03c-68543810cf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:09.359069 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6d7bf2c1-cb69-4e59-b03c-68543810cf62 None None] MFA Rules not processed for user `ef99e1e6bf5f4b7cbd9c702bb479d8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:09.360310 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6d7bf2c1-cb69-4e59-b03c-68543810cf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:09.438282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1361/2704] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:09.451185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94316bd7-c3d4-4d18-813b-cb300925242e admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:09.453088 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-94316bd7-c3d4-4d18-813b-cb300925242e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:09.516675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94316bd7-c3d4-4d18-813b-cb300925242e admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:09.520314 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2cd5f-662a-48a5-92e6-a3aef264ccc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:09.521027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2cd5f-662a-48a5-92e6-a3aef264ccc0 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:09.521356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2cd5f-662a-48a5-92e6-a3aef264ccc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:09.521630 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2cd5f-662a-48a5-92e6-a3aef264ccc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:09.531947 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2cd5f-662a-48a5-92e6-a3aef264ccc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:09.533398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2cd5f-662a-48a5-92e6-a3aef264ccc0 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:09.543436 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1344/2705] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:09.622936 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7bf2c1-cb69-4e59-b03c-68543810cf62 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:09.633109 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7bf2c1-cb69-4e59-b03c-68543810cf62 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:09.636784 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbf803-e18b-47ff-8bf0-b39a664f1798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:09.637424 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbf803-e18b-47ff-8bf0-b39a664f1798 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:09.637699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbf803-e18b-47ff-8bf0-b39a664f1798 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:09.637967 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbf803-e18b-47ff-8bf0-b39a664f1798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:09.823213 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbf803-e18b-47ff-8bf0-b39a664f1798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:09.824693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbf803-e18b-47ff-8bf0-b39a664f1798 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:09.836519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1362/2706] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:10.306418 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2cd5f-662a-48a5-92e6-a3aef264ccc0 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:10.307904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5bd2cd5f-662a-48a5-92e6-a3aef264ccc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:10.384101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2cd5f-662a-48a5-92e6-a3aef264ccc0 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:10.389786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24aaedc-1507-42c6-933c-7eb8e4814df5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:10.390386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aaedc-1507-42c6-933c-7eb8e4814df5 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:10.390661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aaedc-1507-42c6-933c-7eb8e4814df5 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:10.390932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aaedc-1507-42c6-933c-7eb8e4814df5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:10.392692 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e24aaedc-1507-42c6-933c-7eb8e4814df5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:10.405533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24aaedc-1507-42c6-933c-7eb8e4814df5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:10.407117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24aaedc-1507-42c6-933c-7eb8e4814df5 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:10.418868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1345/2707] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:13.551302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbf803-e18b-47ff-8bf0-b39a664f1798 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:13.553397 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ddcbf803-e18b-47ff-8bf0-b39a664f1798 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:13.609599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbf803-e18b-47ff-8bf0-b39a664f1798 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:13.613203 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97b26a-94ef-4e5d-9e3a-16034c4d0f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:13.613802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97b26a-94ef-4e5d-9e3a-16034c4d0f23 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:13.614072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97b26a-94ef-4e5d-9e3a-16034c4d0f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:13.614332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97b26a-94ef-4e5d-9e3a-16034c4d0f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:13.616603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-cb97b26a-94ef-4e5d-9e3a-16034c4d0f23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:13.646547 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:13.693493 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97b26a-94ef-4e5d-9e3a-16034c4d0f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:13.694982 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97b26a-94ef-4e5d-9e3a-16034c4d0f23 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:13.705612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1363/2708] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:17.009694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24aaedc-1507-42c6-933c-7eb8e4814df5 service nova] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:17.011122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e24aaedc-1507-42c6-933c-7eb8e4814df5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:17.077243 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:17.101879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24aaedc-1507-42c6-933c-7eb8e4814df5 service nova] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:17.107950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95d5a2-cf02-4fb2-b467-1e98aca9ab6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08f34295f19b4d6ea405f1e8c8190db5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:17.109235 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95d5a2-cf02-4fb2-b467-1e98aca9ab6a service placement] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:17.109607 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95d5a2-cf02-4fb2-b467-1e98aca9ab6a service placement] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:17.109845 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95d5a2-cf02-4fb2-b467-1e98aca9ab6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:17.111722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ce95d5a2-cf02-4fb2-b467-1e98aca9ab6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:17.125593 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95d5a2-cf02-4fb2-b467-1e98aca9ab6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:17.127381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95d5a2-cf02-4fb2-b467-1e98aca9ab6a service placement] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:17.139446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1346/2709] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:18.915481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97b26a-94ef-4e5d-9e3a-16034c4d0f23 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:18.927594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97b26a-94ef-4e5d-9e3a-16034c4d0f23 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:18.936674 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:18.937788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:18.938237 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:18.938717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] PATH_INFO: `/v3/users/038ff31455104b9e904cf9a3746efdfe` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:18.940482 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=038ff31455104b9e904cf9a3746efdfe)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:18.942945 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:19.011573 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038ff31455104b9e904cf9a3746efdfe'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.026515 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038ff31455104b9e904cf9a3746efdfe'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.034974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038ff31455104b9e904cf9a3746efdfe'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.045462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038ff31455104b9e904cf9a3746efdfe'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.046407 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038ff31455104b9e904cf9a3746efdfe'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.054883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1364/2710] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:18 2022] DELETE /identity/v3/users/038ff31455104b9e904cf9a3746efdfe => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:19.059176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c58064-02c6-41eb-8abb-45ece1a442b0 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:19.060151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c58064-02c6-41eb-8abb-45ece1a442b0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:19.060151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c58064-02c6-41eb-8abb-45ece1a442b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:19.081325 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d4c58064-02c6-41eb-8abb-45ece1a442b0 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:19.083010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d4c58064-02c6-41eb-8abb-45ece1a442b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:19.155632 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1347/2711] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:19.166130 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:19.167608 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:19.244993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e58372-3d16-4604-82c8-6a70b7df70e7 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:19.249813 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d39d2-8795-4ec9-8a2e-9644fd200aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:19.250852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d39d2-8795-4ec9-8a2e-9644fd200aed service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:19.250852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d39d2-8795-4ec9-8a2e-9644fd200aed service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:19.251065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d39d2-8795-4ec9-8a2e-9644fd200aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:19.262367 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d39d2-8795-4ec9-8a2e-9644fd200aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:19.264892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d39d2-8795-4ec9-8a2e-9644fd200aed service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:19.276686 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1365/2712] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:19.583364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c58064-02c6-41eb-8abb-45ece1a442b0 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:19.586124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d4c58064-02c6-41eb-8abb-45ece1a442b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:19.603147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c58064-02c6-41eb-8abb-45ece1a442b0 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:19.609218 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:19.610110 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:19.610452 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:19.610657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] PATH_INFO: `/v3/projects/50e8cb7056d64587a9e3fff212a1373f` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:19.611800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e8cb7056d64587a9e3fff212a1373f)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:19.613838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:19.667842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e8cb7056d64587a9e3fff212a1373f', 'request_id': 'req-c926133d-0f96-4f76-aef5-c1b75c59f4d9'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.676757 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e8cb7056d64587a9e3fff212a1373f', 'request_id': 'req-c926133d-0f96-4f76-aef5-c1b75c59f4d9'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.683775 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e8cb7056d64587a9e3fff212a1373f', 'request_id': 'req-c926133d-0f96-4f76-aef5-c1b75c59f4d9'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.698786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] The token cache is being invalidate because project 50e8cb7056d64587a9e3fff212a1373f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:19.699077 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.699732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.701504 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1348/2713] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:19 2022] DELETE /identity/v3/projects/50e8cb7056d64587a9e3fff212a1373f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:19.705561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d39d2-8795-4ec9-8a2e-9644fd200aed service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:19.717184 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d39d2-8795-4ec9-8a2e-9644fd200aed service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:19.765169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:19.791744 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caef35b-3099-45d7-b6b2-dc7805655e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:19.792414 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caef35b-3099-45d7-b6b2-dc7805655e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:19.792699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caef35b-3099-45d7-b6b2-dc7805655e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:19.792956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caef35b-3099-45d7-b6b2-dc7805655e34 admin admin] PATH_INFO: `/v3/users/ef99e1e6bf5f4b7cbd9c702bb479d8ea` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:19.794102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caef35b-3099-45d7-b6b2-dc7805655e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef99e1e6bf5f4b7cbd9c702bb479d8ea)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:19.796058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caef35b-3099-45d7-b6b2-dc7805655e34 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:19.856200 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6caef35b-3099-45d7-b6b2-dc7805655e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef99e1e6bf5f4b7cbd9c702bb479d8ea'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.876815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6caef35b-3099-45d7-b6b2-dc7805655e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef99e1e6bf5f4b7cbd9c702bb479d8ea'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.887619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6caef35b-3099-45d7-b6b2-dc7805655e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef99e1e6bf5f4b7cbd9c702bb479d8ea'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.897427 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6caef35b-3099-45d7-b6b2-dc7805655e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef99e1e6bf5f4b7cbd9c702bb479d8ea'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.898641 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6caef35b-3099-45d7-b6b2-dc7805655e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef99e1e6bf5f4b7cbd9c702bb479d8ea'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:19.905316 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1366/2714] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:19 2022] DELETE /identity/v3/users/ef99e1e6bf5f4b7cbd9c702bb479d8ea => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:20.229379 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:20.231464 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:20.247747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926133d-0f96-4f76-aef5-c1b75c59f4d9 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:20.315240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68942ac-dbb2-456e-a771-2edb65d272dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:20.316349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68942ac-dbb2-456e-a771-2edb65d272dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:20.316796 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68942ac-dbb2-456e-a771-2edb65d272dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:20.317157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68942ac-dbb2-456e-a771-2edb65d272dd admin admin] PATH_INFO: `/v3/projects/5f0f1ef0450e4f0bbc04d5004e68d917` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:20.318965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68942ac-dbb2-456e-a771-2edb65d272dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f0f1ef0450e4f0bbc04d5004e68d917)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:20.321942 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68942ac-dbb2-456e-a771-2edb65d272dd admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:20.336175 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:20.389938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d68942ac-dbb2-456e-a771-2edb65d272dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f0f1ef0450e4f0bbc04d5004e68d917', 'request_id': 'req-d68942ac-dbb2-456e-a771-2edb65d272dd'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:20.398279 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d68942ac-dbb2-456e-a771-2edb65d272dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f0f1ef0450e4f0bbc04d5004e68d917', 'request_id': 'req-d68942ac-dbb2-456e-a771-2edb65d272dd'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:20.406232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d68942ac-dbb2-456e-a771-2edb65d272dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f0f1ef0450e4f0bbc04d5004e68d917', 'request_id': 'req-d68942ac-dbb2-456e-a771-2edb65d272dd'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:20.424252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d68942ac-dbb2-456e-a771-2edb65d272dd admin admin] The token cache is being invalidate because project 5f0f1ef0450e4f0bbc04d5004e68d917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:20.424797 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d68942ac-dbb2-456e-a771-2edb65d272dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:20.425658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d68942ac-dbb2-456e-a771-2edb65d272dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:20.427255 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1349/2715] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:20 2022] DELETE /identity/v3/projects/5f0f1ef0450e4f0bbc04d5004e68d917 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:20.433481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d32270-9a82-44ab-b21e-c5483c8917e7 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:20.434027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d32270-9a82-44ab-b21e-c5483c8917e7 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:20.434027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d32270-9a82-44ab-b21e-c5483c8917e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:20.458587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-51d32270-9a82-44ab-b21e-c5483c8917e7 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:20.459827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-51d32270-9a82-44ab-b21e-c5483c8917e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:20.591586 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1367/2716] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:20.599211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902b6c1-5613-455b-89cd-6e6ebb6b718b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:20.599508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902b6c1-5613-455b-89cd-6e6ebb6b718b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:20.599750 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902b6c1-5613-455b-89cd-6e6ebb6b718b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:20.634631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-d902b6c1-5613-455b-89cd-6e6ebb6b718b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:20.637734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-d902b6c1-5613-455b-89cd-6e6ebb6b718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:20.647127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1350/2717] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:20.648360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d32270-9a82-44ab-b21e-c5483c8917e7 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:20.649799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-51d32270-9a82-44ab-b21e-c5483c8917e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:20.651236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902b6c1-5613-455b-89cd-6e6ebb6b718b None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:20.663218 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d32270-9a82-44ab-b21e-c5483c8917e7 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:20.664334 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902b6c1-5613-455b-89cd-6e6ebb6b718b None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:20.668895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:20.670240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:20.670679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:20.670955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] PATH_INFO: `/v3/users/2839103f52324a1ab1ebd43fe230addc` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:20.672171 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2839103f52324a1ab1ebd43fe230addc)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:20.674116 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:20.676531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a3d01-fc56-4312-963c-84ef09bfe9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:20.677825 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a3d01-fc56-4312-963c-84ef09bfe9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:20.678251 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a3d01-fc56-4312-963c-84ef09bfe9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:20.678630 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a3d01-fc56-4312-963c-84ef09bfe9a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:20.679972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a3d01-fc56-4312-963c-84ef09bfe9a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:27:20.680558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a3d01-fc56-4312-963c-84ef09bfe9a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:20.682379 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a3d01-fc56-4312-963c-84ef09bfe9a2 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:20.695513 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1351/2718] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:27:20.700034 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a3d01-fc56-4312-963c-84ef09bfe9a2 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:20.718932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a3d01-fc56-4312-963c-84ef09bfe9a2 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:20.726541 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663c746-6b65-418f-91d4-e25d23f6f832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:20.727179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663c746-6b65-418f-91d4-e25d23f6f832 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:20.727534 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663c746-6b65-418f-91d4-e25d23f6f832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:20.727748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663c746-6b65-418f-91d4-e25d23f6f832 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:20.728573 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663c746-6b65-418f-91d4-e25d23f6f832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:20.730479 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663c746-6b65-418f-91d4-e25d23f6f832 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:20.738839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2839103f52324a1ab1ebd43fe230addc'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:20.746240 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:20.756321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2839103f52324a1ab1ebd43fe230addc'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:20.761502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1352/2719] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:20 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:20.765756 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663c746-6b65-418f-91d4-e25d23f6f832 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:20.767128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2839103f52324a1ab1ebd43fe230addc'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:20.775278 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2839103f52324a1ab1ebd43fe230addc'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:20.780101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2839103f52324a1ab1ebd43fe230addc'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:20.790486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1368/2720] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:20 2022] DELETE /identity/v3/users/2839103f52324a1ab1ebd43fe230addc => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:20.829758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663c746-6b65-418f-91d4-e25d23f6f832 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:20.890516 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135faab9-392a-4012-a645-e653eb7e15e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:20.891131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135faab9-392a-4012-a645-e653eb7e15e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:20.891402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135faab9-392a-4012-a645-e653eb7e15e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:20.891664 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135faab9-392a-4012-a645-e653eb7e15e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:20.892324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135faab9-392a-4012-a645-e653eb7e15e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:20.894228 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135faab9-392a-4012-a645-e653eb7e15e4 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:20.903928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:20.920309 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1353/2721] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:20 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:20.924201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:20.925758 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:20.940013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f5748-0b11-4eca-b5c8-b6dc98de2e5e admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:20.944976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b24a29-633b-4382-9ae1-71f61e5331ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:20.945673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b24a29-633b-4382-9ae1-71f61e5331ff admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:20.946038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b24a29-633b-4382-9ae1-71f61e5331ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:20.946380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b24a29-633b-4382-9ae1-71f61e5331ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:20.947160 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b24a29-633b-4382-9ae1-71f61e5331ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:20.949468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b24a29-633b-4382-9ae1-71f61e5331ff admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:20.962854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1369/2722] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:20.966883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135faab9-392a-4012-a645-e653eb7e15e4 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:20.972679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b24a29-633b-4382-9ae1-71f61e5331ff admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:20.974409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d2b24a29-633b-4382-9ae1-71f61e5331ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:20.978309 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135faab9-392a-4012-a645-e653eb7e15e4 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:20.983497 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a754e6f-634d-49a2-8136-c4ac22b34389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:20.984106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a754e6f-634d-49a2-8136-c4ac22b34389 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:20.985102 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a754e6f-634d-49a2-8136-c4ac22b34389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:20.985784 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a754e6f-634d-49a2-8136-c4ac22b34389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:20.986173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a754e6f-634d-49a2-8136-c4ac22b34389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:20.988266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a754e6f-634d-49a2-8136-c4ac22b34389 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:20.988706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b24a29-633b-4382-9ae1-71f61e5331ff admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:20.994293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:20.995038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:20.995384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:20.995737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] PATH_INFO: `/v3/projects/20cdcff29a0448dc92460b2ddd45617b` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:20.997343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20cdcff29a0448dc92460b2ddd45617b)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:20.999388 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:20.999701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1354/2723] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:21.005349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a754e6f-634d-49a2-8136-c4ac22b34389 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:21.016148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a754e6f-634d-49a2-8136-c4ac22b34389 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:21.020973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca1fd3-f38b-4bf4-afbb-30b479c88627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:21.021609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca1fd3-f38b-4bf4-afbb-30b479c88627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:21.021876 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca1fd3-f38b-4bf4-afbb-30b479c88627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:21.022141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca1fd3-f38b-4bf4-afbb-30b479c88627 admin admin] PATH_INFO: `/v3/projects/bb04fc89fb9a4a21bc5152c735913bbd/users/7d7b5664ea9948e6b1b49c2003f8d0ae/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:21.038841 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca1fd3-f38b-4bf4-afbb-30b479c88627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb04fc89fb9a4a21bc5152c735913bbd, user_id=7d7b5664ea9948e6b1b49c2003f8d0ae, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:21.041846 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca1fd3-f38b-4bf4-afbb-30b479c88627 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:21.055196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1355/2724] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:21 2022] PUT /identity/v3/projects/bb04fc89fb9a4a21bc5152c735913bbd/users/7d7b5664ea9948e6b1b49c2003f8d0ae/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:21.056288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20cdcff29a0448dc92460b2ddd45617b', 'request_id': 'req-78063cb2-342d-44cd-8b47-8570a603c330'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.064050 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20cdcff29a0448dc92460b2ddd45617b', 'request_id': 'req-78063cb2-342d-44cd-8b47-8570a603c330'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.065917 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507739fa-59be-42a5-9644-5b571b7e5ddb None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:21.066204 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507739fa-59be-42a5-9644-5b571b7e5ddb None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:21.066474 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507739fa-59be-42a5-9644-5b571b7e5ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:21.070647 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20cdcff29a0448dc92460b2ddd45617b', 'request_id': 'req-78063cb2-342d-44cd-8b47-8570a603c330'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.082751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] The token cache is being invalidate because project 20cdcff29a0448dc92460b2ddd45617b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:21.082966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.083644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.085272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1370/2725] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:20 2022] DELETE /identity/v3/projects/20cdcff29a0448dc92460b2ddd45617b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:21.089289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:21.090610 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:21.095417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-507739fa-59be-42a5-9644-5b571b7e5ddb None None] MFA Rules not processed for user `7d7b5664ea9948e6b1b49c2003f8d0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:21.096578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-507739fa-59be-42a5-9644-5b571b7e5ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:21.158023 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78063cb2-342d-44cd-8b47-8570a603c330 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:21.164427 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1356/2726] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:27:21.169106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507739fa-59be-42a5-9644-5b571b7e5ddb None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:21.170472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-507739fa-59be-42a5-9644-5b571b7e5ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:21.207852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:21.223539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:21.224318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:21.224695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:21.225079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] PATH_INFO: `/v3/users/b5398b12577a41cb9891845bc3671f95` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:21.227730 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5398b12577a41cb9891845bc3671f95)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:21.230328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:21.238027 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507739fa-59be-42a5-9644-5b571b7e5ddb None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:21.243284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65735c-8036-4043-9ce3-171c327af1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:21.243985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65735c-8036-4043-9ce3-171c327af1ca admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:21.244560 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65735c-8036-4043-9ce3-171c327af1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:21.244835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65735c-8036-4043-9ce3-171c327af1ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:21.245523 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65735c-8036-4043-9ce3-171c327af1ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:21.247446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65735c-8036-4043-9ce3-171c327af1ca admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:21.279643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1357/2727] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:21 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:21.280709 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65735c-8036-4043-9ce3-171c327af1ca admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:21.280843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5398b12577a41cb9891845bc3671f95'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.298974 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5398b12577a41cb9891845bc3671f95'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.306070 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5398b12577a41cb9891845bc3671f95'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.313504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5398b12577a41cb9891845bc3671f95'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.314285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5398b12577a41cb9891845bc3671f95'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.318287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:21.321725 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1371/2728] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:21 2022] DELETE /identity/v3/users/b5398b12577a41cb9891845bc3671f95 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:21.352271 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65735c-8036-4043-9ce3-171c327af1ca admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:21.411171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869de264-247b-493a-98cc-1b9462b0e088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:21.411792 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869de264-247b-493a-98cc-1b9462b0e088 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:21.412345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869de264-247b-493a-98cc-1b9462b0e088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:21.412345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869de264-247b-493a-98cc-1b9462b0e088 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:21.412992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869de264-247b-493a-98cc-1b9462b0e088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:21.415129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869de264-247b-493a-98cc-1b9462b0e088 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:21.440536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1358/2729] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:21 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:21.441819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:21.443251 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:21.456975 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eae29d-cf4b-4915-99ed-ba058839a694 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:21.462321 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f90e48-b870-4336-9445-00fb60c3bae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:21.462993 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f90e48-b870-4336-9445-00fb60c3bae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:21.463333 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f90e48-b870-4336-9445-00fb60c3bae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:21.463662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f90e48-b870-4336-9445-00fb60c3bae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:21.464344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f90e48-b870-4336-9445-00fb60c3bae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:21.466029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f90e48-b870-4336-9445-00fb60c3bae8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:21.476676 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1372/2730] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:21.480734 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869de264-247b-493a-98cc-1b9462b0e088 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:21.492794 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869de264-247b-493a-98cc-1b9462b0e088 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:21.498344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddbbd7-af59-4f82-bb29-425b9871c249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:21.502089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddbbd7-af59-4f82-bb29-425b9871c249 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:21.502089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddbbd7-af59-4f82-bb29-425b9871c249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:21.502089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddbbd7-af59-4f82-bb29-425b9871c249 admin admin] PATH_INFO: `/v3/projects/ce862410789142aca08a6515dedc054a/users/f0ee31bba93b4295a82a0b50afdffc87/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:21.503463 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f90e48-b870-4336-9445-00fb60c3bae8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:21.505464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-83f90e48-b870-4336-9445-00fb60c3bae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:21.518120 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ddbbd7-af59-4f82-bb29-425b9871c249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce862410789142aca08a6515dedc054a, user_id=f0ee31bba93b4295a82a0b50afdffc87, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:21.520914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ddbbd7-af59-4f82-bb29-425b9871c249 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:21.528911 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f90e48-b870-4336-9445-00fb60c3bae8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:21.535181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:21.535916 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:21.536341 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:21.536792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] PATH_INFO: `/v3/projects/c1519945e9154f16adf095a4e25a067d` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:21.537442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1359/2731] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:21 2022] PUT /identity/v3/projects/ce862410789142aca08a6515dedc054a/users/f0ee31bba93b4295a82a0b50afdffc87/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:21.538052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1519945e9154f16adf095a4e25a067d)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:21.541075 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:21.541381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddbbd7-af59-4f82-bb29-425b9871c249 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:21.594912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1519945e9154f16adf095a4e25a067d', 'request_id': 'req-24643740-f146-4f21-8c4a-bfa306b7b084'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.604320 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1519945e9154f16adf095a4e25a067d', 'request_id': 'req-24643740-f146-4f21-8c4a-bfa306b7b084'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.611962 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1519945e9154f16adf095a4e25a067d', 'request_id': 'req-24643740-f146-4f21-8c4a-bfa306b7b084'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.614487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddbbd7-af59-4f82-bb29-425b9871c249 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:21.629388 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] The token cache is being invalidate because project c1519945e9154f16adf095a4e25a067d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:21.629679 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.630396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:21.631892 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1373/2732] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:21 2022] DELETE /identity/v3/projects/c1519945e9154f16adf095a4e25a067d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:21.668159 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:21.681252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797e6b4-a810-44d2-a0d5-3f80cd708651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:21.681959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e6b4-a810-44d2-a0d5-3f80cd708651 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:21.682234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e6b4-a810-44d2-a0d5-3f80cd708651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:21.682501 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e6b4-a810-44d2-a0d5-3f80cd708651 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:21.683185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797e6b4-a810-44d2-a0d5-3f80cd708651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:21.689292 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797e6b4-a810-44d2-a0d5-3f80cd708651 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:21.696280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1360/2733] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:27:21.700775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:21.702343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:21.720379 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24643740-f146-4f21-8c4a-bfa306b7b084 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:21.725255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26d03f-1c6d-46e7-bbd5-8bb052ec1dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:21.726693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d03f-1c6d-46e7-bbd5-8bb052ec1dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:21.727158 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d03f-1c6d-46e7-bbd5-8bb052ec1dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:21.727520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d03f-1c6d-46e7-bbd5-8bb052ec1dd3 admin admin] PATH_INFO: `/v3/projects/ce862410789142aca08a6515dedc054a/users/f0ee31bba93b4295a82a0b50afdffc87/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:21.729661 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26d03f-1c6d-46e7-bbd5-8bb052ec1dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce862410789142aca08a6515dedc054a, user_id=f0ee31bba93b4295a82a0b50afdffc87, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:21.732469 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26d03f-1c6d-46e7-bbd5-8bb052ec1dd3 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:21.744526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1374/2734] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:21 2022] PUT /identity/v3/projects/ce862410789142aca08a6515dedc054a/users/f0ee31bba93b4295a82a0b50afdffc87/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:21.818009 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fe9e1-0f2a-4df6-895d-6e0fdb610bfa None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:21.818270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fe9e1-0f2a-4df6-895d-6e0fdb610bfa None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:21.818570 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fe9e1-0f2a-4df6-895d-6e0fdb610bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:21.854141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8c3fe9e1-0f2a-4df6-895d-6e0fdb610bfa None None] MFA Rules not processed for user `f0ee31bba93b4295a82a0b50afdffc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:21.855326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8c3fe9e1-0f2a-4df6-895d-6e0fdb610bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:21.931877 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1361/2735] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:27:21.943137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26d03f-1c6d-46e7-bbd5-8bb052ec1dd3 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:21.944970 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9d26d03f-1c6d-46e7-bbd5-8bb052ec1dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:21.977267 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:22.004549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26d03f-1c6d-46e7-bbd5-8bb052ec1dd3 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:22.008759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdc4a5-4843-49c7-9f2c-7ebcd09306bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:22.009357 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdc4a5-4843-49c7-9f2c-7ebcd09306bb service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:22.009626 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdc4a5-4843-49c7-9f2c-7ebcd09306bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:22.009889 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdc4a5-4843-49c7-9f2c-7ebcd09306bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:22.018583 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bdc4a5-4843-49c7-9f2c-7ebcd09306bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:22.020021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bdc4a5-4843-49c7-9f2c-7ebcd09306bb service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:22.030294 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1375/2736] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:22.460173 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fe9e1-0f2a-4df6-895d-6e0fdb610bfa None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:22.525829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fe9e1-0f2a-4df6-895d-6e0fdb610bfa None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:22.530287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:22.530904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:22.531177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:22.531441 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] PATH_INFO: `/v3/users/7d7b5664ea9948e6b1b49c2003f8d0ae` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:22.532556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d7b5664ea9948e6b1b49c2003f8d0ae)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:22.534437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:22.581705 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d7b5664ea9948e6b1b49c2003f8d0ae'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:22.588965 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d7b5664ea9948e6b1b49c2003f8d0ae'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:22.589604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d7b5664ea9948e6b1b49c2003f8d0ae'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:22.590387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d7b5664ea9948e6b1b49c2003f8d0ae'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:22.603028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d7b5664ea9948e6b1b49c2003f8d0ae'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:22.604984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:22.618326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1362/2737] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:22 2022] DELETE /identity/v3/users/7d7b5664ea9948e6b1b49c2003f8d0ae => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:22.622981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35bf24-4ab1-4ce7-a061-8e8ad854c184 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:22.623479 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35bf24-4ab1-4ce7-a061-8e8ad854c184 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:22.623599 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35bf24-4ab1-4ce7-a061-8e8ad854c184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:22.644423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-9a35bf24-4ab1-4ce7-a061-8e8ad854c184 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:22.645637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9a35bf24-4ab1-4ce7-a061-8e8ad854c184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:22.706756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1376/2738] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:22.717240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:22.718679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:22.778258 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dfee3-1fb1-4b74-9a19-0db02583e315 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:22.782357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa70bd-e699-4948-846f-9a3d29d5fc25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:22.782964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa70bd-e699-4948-846f-9a3d29d5fc25 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:22.783234 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa70bd-e699-4948-846f-9a3d29d5fc25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:22.783505 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa70bd-e699-4948-846f-9a3d29d5fc25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:22.794056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa70bd-e699-4948-846f-9a3d29d5fc25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:22.795475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa70bd-e699-4948-846f-9a3d29d5fc25 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:22.805735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1363/2739] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:23.074293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35bf24-4ab1-4ce7-a061-8e8ad854c184 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:23.075777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-9a35bf24-4ab1-4ce7-a061-8e8ad854c184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:23.088215 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35bf24-4ab1-4ce7-a061-8e8ad854c184 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:23.092907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:23.093542 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:23.093822 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:23.094091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] PATH_INFO: `/v3/projects/bb04fc89fb9a4a21bc5152c735913bbd` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:23.095223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb04fc89fb9a4a21bc5152c735913bbd)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:23.097157 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:23.147392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb04fc89fb9a4a21bc5152c735913bbd', 'request_id': 'req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.155449 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb04fc89fb9a4a21bc5152c735913bbd', 'request_id': 'req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.161127 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb04fc89fb9a4a21bc5152c735913bbd', 'request_id': 'req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.173145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] The token cache is being invalidate because project bb04fc89fb9a4a21bc5152c735913bbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:23.173426 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.174076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.175580 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1377/2740] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:23 2022] DELETE /identity/v3/projects/bb04fc89fb9a4a21bc5152c735913bbd => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:23.180155 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa70bd-e699-4948-846f-9a3d29d5fc25 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:23.189841 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa70bd-e699-4948-846f-9a3d29d5fc25 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:23.248331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aad0b0-37b1-4df3-8959-0f502181c554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:23.249036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aad0b0-37b1-4df3-8959-0f502181c554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:23.249309 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aad0b0-37b1-4df3-8959-0f502181c554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:23.249585 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aad0b0-37b1-4df3-8959-0f502181c554 admin admin] PATH_INFO: `/v3/users/f0ee31bba93b4295a82a0b50afdffc87` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:23.250714 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aad0b0-37b1-4df3-8959-0f502181c554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0ee31bba93b4295a82a0b50afdffc87)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:23.252671 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aad0b0-37b1-4df3-8959-0f502181c554 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:23.299028 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-37aad0b0-37b1-4df3-8959-0f502181c554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0ee31bba93b4295a82a0b50afdffc87'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.307017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-37aad0b0-37b1-4df3-8959-0f502181c554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0ee31bba93b4295a82a0b50afdffc87'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.307636 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-37aad0b0-37b1-4df3-8959-0f502181c554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0ee31bba93b4295a82a0b50afdffc87'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.308259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-37aad0b0-37b1-4df3-8959-0f502181c554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0ee31bba93b4295a82a0b50afdffc87'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.320272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-37aad0b0-37b1-4df3-8959-0f502181c554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0ee31bba93b4295a82a0b50afdffc87'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.331901 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1364/2741] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:23 2022] DELETE /identity/v3/users/f0ee31bba93b4295a82a0b50afdffc87 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:23.605552 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:23.607097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:23.621614 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ffedf-e5ac-45a2-b13f-21edc5f8cb52 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:23.674332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:23.682606 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc646028-5684-47d6-a07b-25600ac6df51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:23.683333 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc646028-5684-47d6-a07b-25600ac6df51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:23.683830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc646028-5684-47d6-a07b-25600ac6df51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:23.684179 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc646028-5684-47d6-a07b-25600ac6df51 admin admin] PATH_INFO: `/v3/projects/ce862410789142aca08a6515dedc054a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:23.685491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc646028-5684-47d6-a07b-25600ac6df51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce862410789142aca08a6515dedc054a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:23.687539 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc646028-5684-47d6-a07b-25600ac6df51 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:23.740124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-fc646028-5684-47d6-a07b-25600ac6df51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce862410789142aca08a6515dedc054a', 'request_id': 'req-fc646028-5684-47d6-a07b-25600ac6df51'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.748283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-fc646028-5684-47d6-a07b-25600ac6df51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce862410789142aca08a6515dedc054a', 'request_id': 'req-fc646028-5684-47d6-a07b-25600ac6df51'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.753955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-fc646028-5684-47d6-a07b-25600ac6df51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce862410789142aca08a6515dedc054a', 'request_id': 'req-fc646028-5684-47d6-a07b-25600ac6df51'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.765965 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-fc646028-5684-47d6-a07b-25600ac6df51 admin admin] The token cache is being invalidate because project ce862410789142aca08a6515dedc054a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:23.766255 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-fc646028-5684-47d6-a07b-25600ac6df51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.766901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-fc646028-5684-47d6-a07b-25600ac6df51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:23.768548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1378/2742] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:23 2022] DELETE /identity/v3/projects/ce862410789142aca08a6515dedc054a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:23.774109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf978a6-44b1-4f4d-b50b-5a080ba1a9a0 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:23.774574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf978a6-44b1-4f4d-b50b-5a080ba1a9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:23.774749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf978a6-44b1-4f4d-b50b-5a080ba1a9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:23.794771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-ecf978a6-44b1-4f4d-b50b-5a080ba1a9a0 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:23.795972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ecf978a6-44b1-4f4d-b50b-5a080ba1a9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:23.804465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:23.861313 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1365/2743] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:23.870010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723d4ab-21c6-4044-8a2e-1cb33e951af3 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:23.870293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723d4ab-21c6-4044-8a2e-1cb33e951af3 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:23.870563 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723d4ab-21c6-4044-8a2e-1cb33e951af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:23.890717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-4723d4ab-21c6-4044-8a2e-1cb33e951af3 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:23.891921 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4723d4ab-21c6-4044-8a2e-1cb33e951af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:23.899964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1379/2744] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:23.904387 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf978a6-44b1-4f4d-b50b-5a080ba1a9a0 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:23.914030 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf978a6-44b1-4f4d-b50b-5a080ba1a9a0 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:23.918531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac412c4-704b-415d-bc93-f618335eb2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:23.919122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac412c4-704b-415d-bc93-f618335eb2bd admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:23.919393 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac412c4-704b-415d-bc93-f618335eb2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:23.919658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac412c4-704b-415d-bc93-f618335eb2bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:23.920323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac412c4-704b-415d-bc93-f618335eb2bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:27:23.920619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac412c4-704b-415d-bc93-f618335eb2bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:23.922186 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac412c4-704b-415d-bc93-f618335eb2bd admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:23.933904 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1366/2745] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:27:23.938207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723d4ab-21c6-4044-8a2e-1cb33e951af3 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:23.948254 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723d4ab-21c6-4044-8a2e-1cb33e951af3 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:23.953000 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae0e29-d812-49d2-b4a2-6b515a8b18bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:23.953603 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae0e29-d812-49d2-b4a2-6b515a8b18bd admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:23.953882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae0e29-d812-49d2-b4a2-6b515a8b18bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:23.954154 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae0e29-d812-49d2-b4a2-6b515a8b18bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:23.954787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae0e29-d812-49d2-b4a2-6b515a8b18bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:23.956735 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae0e29-d812-49d2-b4a2-6b515a8b18bd admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:23.983576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1380/2746] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:23 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:23.987831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac412c4-704b-415d-bc93-f618335eb2bd admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:23.998928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac412c4-704b-415d-bc93-f618335eb2bd admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:24.053495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a84ba5-10c7-460b-a070-599dc1335c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:24.054187 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a84ba5-10c7-460b-a070-599dc1335c13 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.055645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a84ba5-10c7-460b-a070-599dc1335c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.055645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a84ba5-10c7-460b-a070-599dc1335c13 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.056095 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a84ba5-10c7-460b-a070-599dc1335c13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:24.057918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a84ba5-10c7-460b-a070-599dc1335c13 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:24.079083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1367/2747] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:23 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:24.083210 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae0e29-d812-49d2-b4a2-6b515a8b18bd admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:24.093887 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae0e29-d812-49d2-b4a2-6b515a8b18bd admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:24.098775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75471e-0837-4f52-adce-aba5150de55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:24.099383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75471e-0837-4f52-adce-aba5150de55c admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.099662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75471e-0837-4f52-adce-aba5150de55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.099934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75471e-0837-4f52-adce-aba5150de55c admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.100612 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75471e-0837-4f52-adce-aba5150de55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:24.102282 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75471e-0837-4f52-adce-aba5150de55c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:24.113495 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1381/2748] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:24.118123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a84ba5-10c7-460b-a070-599dc1335c13 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:24.128771 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a84ba5-10c7-460b-a070-599dc1335c13 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:24.134129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c1d92-438f-4e5a-8d02-e9b40eb8ccd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:24.134718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c1d92-438f-4e5a-8d02-e9b40eb8ccd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.134992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c1d92-438f-4e5a-8d02-e9b40eb8ccd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.135259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c1d92-438f-4e5a-8d02-e9b40eb8ccd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.135899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c1d92-438f-4e5a-8d02-e9b40eb8ccd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:24.137504 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c1d92-438f-4e5a-8d02-e9b40eb8ccd0 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:24.147861 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1368/2749] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:24.152380 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75471e-0837-4f52-adce-aba5150de55c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:24.162343 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75471e-0837-4f52-adce-aba5150de55c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:24.167231 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1005fe3-2715-42f2-80f4-1753da9471cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:24.167885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1005fe3-2715-42f2-80f4-1753da9471cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.168183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1005fe3-2715-42f2-80f4-1753da9471cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.169989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1005fe3-2715-42f2-80f4-1753da9471cb admin admin] PATH_INFO: `/v3/projects/8f9f300556804efe8037fc0630e5e80c/users/7ff0ad91dac84acea2c163e75866c3c3/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.184123 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1005fe3-2715-42f2-80f4-1753da9471cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9f300556804efe8037fc0630e5e80c, user_id=7ff0ad91dac84acea2c163e75866c3c3, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:24.186724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1005fe3-2715-42f2-80f4-1753da9471cb admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:24.200222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1382/2750] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:24 2022] PUT /identity/v3/projects/8f9f300556804efe8037fc0630e5e80c/users/7ff0ad91dac84acea2c163e75866c3c3/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:24.210267 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bdc40-49f5-4f0a-a7e9-bd94af75f177 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.210545 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bdc40-49f5-4f0a-a7e9-bd94af75f177 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.210840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bdc40-49f5-4f0a-a7e9-bd94af75f177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.238101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-084bdc40-49f5-4f0a-a7e9-bd94af75f177 None None] MFA Rules not processed for user `7ff0ad91dac84acea2c163e75866c3c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:24.239282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-084bdc40-49f5-4f0a-a7e9-bd94af75f177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:24.303006 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1369/2751] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:27:24.308256 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1005fe3-2715-42f2-80f4-1753da9471cb admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:24.367932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1005fe3-2715-42f2-80f4-1753da9471cb admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:24.372881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f7ea1-c238-4916-8c41-a880b495ab13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:24.373500 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f7ea1-c238-4916-8c41-a880b495ab13 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.373785 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f7ea1-c238-4916-8c41-a880b495ab13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.374058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f7ea1-c238-4916-8c41-a880b495ab13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.374690 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f7ea1-c238-4916-8c41-a880b495ab13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:24.376621 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f7ea1-c238-4916-8c41-a880b495ab13 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:24.407165 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1383/2752] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:24 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:24.409116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bdc40-49f5-4f0a-a7e9-bd94af75f177 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:24.423763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bdc40-49f5-4f0a-a7e9-bd94af75f177 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:24.429263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:24.484305 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd4007-7434-4d37-b2b7-838c3b2253b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:24.484828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd4007-7434-4d37-b2b7-838c3b2253b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.485115 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd4007-7434-4d37-b2b7-838c3b2253b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.485380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd4007-7434-4d37-b2b7-838c3b2253b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.486019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd4007-7434-4d37-b2b7-838c3b2253b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:24.487923 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd4007-7434-4d37-b2b7-838c3b2253b7 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:24.511266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1370/2753] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:24 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:24.514946 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f7ea1-c238-4916-8c41-a880b495ab13 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:24.525856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f7ea1-c238-4916-8c41-a880b495ab13 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:24.530642 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee709e0-30e5-4570-87c9-17cdeecf65a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:24.531241 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee709e0-30e5-4570-87c9-17cdeecf65a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.531520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee709e0-30e5-4570-87c9-17cdeecf65a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.531799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee709e0-30e5-4570-87c9-17cdeecf65a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.532480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee709e0-30e5-4570-87c9-17cdeecf65a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:24.534072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee709e0-30e5-4570-87c9-17cdeecf65a5 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:24.535325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:24.550698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1384/2754] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:24.551931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd4007-7434-4d37-b2b7-838c3b2253b7 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:24.562524 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd4007-7434-4d37-b2b7-838c3b2253b7 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:24.567178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a8261-5722-46b7-9f98-6f63afd0555e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:24.567816 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a8261-5722-46b7-9f98-6f63afd0555e admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.568126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a8261-5722-46b7-9f98-6f63afd0555e admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.568398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a8261-5722-46b7-9f98-6f63afd0555e admin admin] PATH_INFO: `/v3/projects/a3e3811ab5ad46db8e6577d81c817ae7/users/37fa68fe28b34f06871ed301431c9976/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.583921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a8261-5722-46b7-9f98-6f63afd0555e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e3811ab5ad46db8e6577d81c817ae7, user_id=37fa68fe28b34f06871ed301431c9976, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:24.587056 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a8261-5722-46b7-9f98-6f63afd0555e admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:24.599710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1371/2755] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:24 2022] PUT /identity/v3/projects/a3e3811ab5ad46db8e6577d81c817ae7/users/37fa68fe28b34f06871ed301431c9976/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:24.604646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee709e0-30e5-4570-87c9-17cdeecf65a5 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:24.615280 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee709e0-30e5-4570-87c9-17cdeecf65a5 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:24.672076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbea2a9-af82-47c7-901c-63e4b339c8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:24.672682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbea2a9-af82-47c7-901c-63e4b339c8cd admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.672957 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbea2a9-af82-47c7-901c-63e4b339c8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.673223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbea2a9-af82-47c7-901c-63e4b339c8cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.673859 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbea2a9-af82-47c7-901c-63e4b339c8cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:24.675722 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbea2a9-af82-47c7-901c-63e4b339c8cd admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:24.692940 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1385/2756] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:24.694216 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a8261-5722-46b7-9f98-6f63afd0555e admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:24.704899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a8261-5722-46b7-9f98-6f63afd0555e admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:24.710052 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd20d7-4ced-4bbe-99f0-92db51fd439f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:24.710748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd20d7-4ced-4bbe-99f0-92db51fd439f admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.711613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd20d7-4ced-4bbe-99f0-92db51fd439f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.711885 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd20d7-4ced-4bbe-99f0-92db51fd439f admin admin] PATH_INFO: `/v3/projects/a3e3811ab5ad46db8e6577d81c817ae7/users/37fa68fe28b34f06871ed301431c9976/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.714007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd20d7-4ced-4bbe-99f0-92db51fd439f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e3811ab5ad46db8e6577d81c817ae7, user_id=37fa68fe28b34f06871ed301431c9976, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:24.716630 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd20d7-4ced-4bbe-99f0-92db51fd439f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:24.730684 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1372/2757] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:24 2022] PUT /identity/v3/projects/a3e3811ab5ad46db8e6577d81c817ae7/users/37fa68fe28b34f06871ed301431c9976/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:24.742575 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61dc93-0561-41a7-992a-a1bd02c7af3d None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.742855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61dc93-0561-41a7-992a-a1bd02c7af3d None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.743101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61dc93-0561-41a7-992a-a1bd02c7af3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.772801 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7f61dc93-0561-41a7-992a-a1bd02c7af3d None None] MFA Rules not processed for user `37fa68fe28b34f06871ed301431c9976`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:24.774585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7f61dc93-0561-41a7-992a-a1bd02c7af3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:24.846145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1386/2758] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:24.857820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd20d7-4ced-4bbe-99f0-92db51fd439f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:24.859873 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-85cd20d7-4ced-4bbe-99f0-92db51fd439f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:24.921540 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd20d7-4ced-4bbe-99f0-92db51fd439f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:24.925132 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906a51b-e7d8-43fe-9b33-35f293ca4d22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:24.925732 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906a51b-e7d8-43fe-9b33-35f293ca4d22 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:24.926005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906a51b-e7d8-43fe-9b33-35f293ca4d22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:24.926272 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906a51b-e7d8-43fe-9b33-35f293ca4d22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:24.935126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906a51b-e7d8-43fe-9b33-35f293ca4d22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:24.936577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906a51b-e7d8-43fe-9b33-35f293ca4d22 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:24.946311 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1373/2759] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:24.990021 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61dc93-0561-41a7-992a-a1bd02c7af3d None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:24.991393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7f61dc93-0561-41a7-992a-a1bd02c7af3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:25.003694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61dc93-0561-41a7-992a-a1bd02c7af3d None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:25.007334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42641e8-7b9b-4c44-8aa4-5668b2ad25e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:25.008501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42641e8-7b9b-4c44-8aa4-5668b2ad25e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:25.008856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42641e8-7b9b-4c44-8aa4-5668b2ad25e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:25.009118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42641e8-7b9b-4c44-8aa4-5668b2ad25e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:25.064877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42641e8-7b9b-4c44-8aa4-5668b2ad25e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:25.066318 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42641e8-7b9b-4c44-8aa4-5668b2ad25e6 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:25.076249 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1387/2760] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:28.403937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906a51b-e7d8-43fe-9b33-35f293ca4d22 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:28.466435 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906a51b-e7d8-43fe-9b33-35f293ca4d22 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:28.471015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:28.471631 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:28.471905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:28.472198 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] PATH_INFO: `/v3/users/7ff0ad91dac84acea2c163e75866c3c3` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:28.473297 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ff0ad91dac84acea2c163e75866c3c3)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:28.475178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:28.480440 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:28.522980 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ff0ad91dac84acea2c163e75866c3c3'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:28.530794 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ff0ad91dac84acea2c163e75866c3c3'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:28.531405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ff0ad91dac84acea2c163e75866c3c3'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:28.532060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ff0ad91dac84acea2c163e75866c3c3'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:28.544569 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ff0ad91dac84acea2c163e75866c3c3'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:28.556974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1374/2761] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:28 2022] DELETE /identity/v3/users/7ff0ad91dac84acea2c163e75866c3c3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:28.561530 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383a3f-e99d-4ec1-a1d5-8ff79955abb5 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:28.562019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383a3f-e99d-4ec1-a1d5-8ff79955abb5 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:28.562161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383a3f-e99d-4ec1-a1d5-8ff79955abb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:28.581276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-a7383a3f-e99d-4ec1-a1d5-8ff79955abb5 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:28.582470 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a7383a3f-e99d-4ec1-a1d5-8ff79955abb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:28.592130 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=100 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:28.643209 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1388/2762] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:28.653349 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:28.654758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:28.715896 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754fd40-752d-420c-918a-6768b4f2e182 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:28.719539 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413cf57-98a5-4d66-bd20-fb014a2a430e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:28.720185 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413cf57-98a5-4d66-bd20-fb014a2a430e service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:28.720485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413cf57-98a5-4d66-bd20-fb014a2a430e service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:28.720868 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413cf57-98a5-4d66-bd20-fb014a2a430e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:28.731043 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413cf57-98a5-4d66-bd20-fb014a2a430e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:28.732501 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413cf57-98a5-4d66-bd20-fb014a2a430e service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:28.742287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1375/2763] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:27:29.092291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7383a3f-e99d-4ec1-a1d5-8ff79955abb5 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:29.093830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-a7383a3f-e99d-4ec1-a1d5-8ff79955abb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:29.105912 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7383a3f-e99d-4ec1-a1d5-8ff79955abb5 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:29.111643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:29.112554 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:29.112851 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:29.113159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] PATH_INFO: `/v3/projects/8f9f300556804efe8037fc0630e5e80c` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:29.114394 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9f300556804efe8037fc0630e5e80c)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:29.116458 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:29.171480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9f300556804efe8037fc0630e5e80c', 'request_id': 'req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.179870 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9f300556804efe8037fc0630e5e80c', 'request_id': 'req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.186971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9f300556804efe8037fc0630e5e80c', 'request_id': 'req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.199435 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] The token cache is being invalidate because project 8f9f300556804efe8037fc0630e5e80c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:29.199831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.200433 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.202167 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1389/2764] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:29 2022] DELETE /identity/v3/projects/8f9f300556804efe8037fc0630e5e80c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:29.205996 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413cf57-98a5-4d66-bd20-fb014a2a430e service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:29.216087 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413cf57-98a5-4d66-bd20-fb014a2a430e service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:29.279397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c608f-7116-4f11-b8dd-212cdae17713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:29.280059 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c608f-7116-4f11-b8dd-212cdae17713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:29.280339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c608f-7116-4f11-b8dd-212cdae17713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:29.280609 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c608f-7116-4f11-b8dd-212cdae17713 admin admin] PATH_INFO: `/v3/users/37fa68fe28b34f06871ed301431c9976` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:29.281733 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c608f-7116-4f11-b8dd-212cdae17713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37fa68fe28b34f06871ed301431c9976)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:29.283743 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c608f-7116-4f11-b8dd-212cdae17713 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:29.313657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:29.335323 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-195c608f-7116-4f11-b8dd-212cdae17713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37fa68fe28b34f06871ed301431c9976'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.343190 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-195c608f-7116-4f11-b8dd-212cdae17713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37fa68fe28b34f06871ed301431c9976'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.343842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-195c608f-7116-4f11-b8dd-212cdae17713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37fa68fe28b34f06871ed301431c9976'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.344583 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-195c608f-7116-4f11-b8dd-212cdae17713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37fa68fe28b34f06871ed301431c9976'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.357531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-195c608f-7116-4f11-b8dd-212cdae17713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37fa68fe28b34f06871ed301431c9976'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.370811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1376/2765] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:29 2022] DELETE /identity/v3/users/37fa68fe28b34f06871ed301431c9976 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:29.663971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:29.665881 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:29.681891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041fe35-9bcb-40c5-943f-819c1fa6dcc1 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:29.747474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c3859-6509-41f5-95f6-d5f5d947f722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:29.748086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c3859-6509-41f5-95f6-d5f5d947f722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:29.748342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c3859-6509-41f5-95f6-d5f5d947f722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:29.748616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c3859-6509-41f5-95f6-d5f5d947f722 admin admin] PATH_INFO: `/v3/projects/a3e3811ab5ad46db8e6577d81c817ae7` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:29.749854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c3859-6509-41f5-95f6-d5f5d947f722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e3811ab5ad46db8e6577d81c817ae7)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:29.752335 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c3859-6509-41f5-95f6-d5f5d947f722 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:29.796065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:29.804847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-056c3859-6509-41f5-95f6-d5f5d947f722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e3811ab5ad46db8e6577d81c817ae7', 'request_id': 'req-056c3859-6509-41f5-95f6-d5f5d947f722'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.813600 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-056c3859-6509-41f5-95f6-d5f5d947f722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e3811ab5ad46db8e6577d81c817ae7', 'request_id': 'req-056c3859-6509-41f5-95f6-d5f5d947f722'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.820396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-056c3859-6509-41f5-95f6-d5f5d947f722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e3811ab5ad46db8e6577d81c817ae7', 'request_id': 'req-056c3859-6509-41f5-95f6-d5f5d947f722'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.832197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-056c3859-6509-41f5-95f6-d5f5d947f722 admin admin] The token cache is being invalidate because project a3e3811ab5ad46db8e6577d81c817ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:29.832481 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-056c3859-6509-41f5-95f6-d5f5d947f722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.833135 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-056c3859-6509-41f5-95f6-d5f5d947f722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:29.834807 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1390/2766] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:29 2022] DELETE /identity/v3/projects/a3e3811ab5ad46db8e6577d81c817ae7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:29.840123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b88cf-a679-4564-8c3c-9a36ea53f08c None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:29.840639 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b88cf-a679-4564-8c3c-9a36ea53f08c None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:29.840795 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b88cf-a679-4564-8c3c-9a36ea53f08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:29.869183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-1c2b88cf-a679-4564-8c3c-9a36ea53f08c None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:29.870849 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-1c2b88cf-a679-4564-8c3c-9a36ea53f08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:29.939465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1377/2767] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:29.948124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e175ffc-2487-4ab4-8592-d619f7fb8962 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:29.948461 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e175ffc-2487-4ab4-8592-d619f7fb8962 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:29.948660 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e175ffc-2487-4ab4-8592-d619f7fb8962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:29.970468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-6e175ffc-2487-4ab4-8592-d619f7fb8962 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:29.971667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6e175ffc-2487-4ab4-8592-d619f7fb8962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:29.979694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1391/2768] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:29.983958 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b88cf-a679-4564-8c3c-9a36ea53f08c None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:29.999698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b88cf-a679-4564-8c3c-9a36ea53f08c None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:30.005310 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99aac4f-3e55-4de4-8af6-4a2b657d8136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:30.005859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99aac4f-3e55-4de4-8af6-4a2b657d8136 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:30.006131 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99aac4f-3e55-4de4-8af6-4a2b657d8136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:30.006446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99aac4f-3e55-4de4-8af6-4a2b657d8136 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:30.007088 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99aac4f-3e55-4de4-8af6-4a2b657d8136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:27:30.007378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99aac4f-3e55-4de4-8af6-4a2b657d8136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:30.009578 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99aac4f-3e55-4de4-8af6-4a2b657d8136 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:30.023445 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1378/2769] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:27:30.028741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e175ffc-2487-4ab4-8592-d619f7fb8962 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:30.039925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e175ffc-2487-4ab4-8592-d619f7fb8962 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:30.045484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80ed1b-20fd-4a6f-8263-b299bca92af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:30.046153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80ed1b-20fd-4a6f-8263-b299bca92af8 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:30.046497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80ed1b-20fd-4a6f-8263-b299bca92af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:30.046825 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80ed1b-20fd-4a6f-8263-b299bca92af8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:30.048802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80ed1b-20fd-4a6f-8263-b299bca92af8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:30.050764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80ed1b-20fd-4a6f-8263-b299bca92af8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:30.079667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1392/2770] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:30 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:30.083831 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99aac4f-3e55-4de4-8af6-4a2b657d8136 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:30.094646 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99aac4f-3e55-4de4-8af6-4a2b657d8136 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:30.158510 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689eb12-74a7-4566-965b-737419827134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:30.159117 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689eb12-74a7-4566-965b-737419827134 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:30.159391 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689eb12-74a7-4566-965b-737419827134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:30.159661 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689eb12-74a7-4566-965b-737419827134 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:30.160345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689eb12-74a7-4566-965b-737419827134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:30.162289 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689eb12-74a7-4566-965b-737419827134 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:30.187808 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1379/2771] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:30 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:30.192189 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80ed1b-20fd-4a6f-8263-b299bca92af8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:30.203809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80ed1b-20fd-4a6f-8263-b299bca92af8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:30.208588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e22f2-4f55-41f0-be90-2408ab990701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:30.209286 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e22f2-4f55-41f0-be90-2408ab990701 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:30.209471 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e22f2-4f55-41f0-be90-2408ab990701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:30.209741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e22f2-4f55-41f0-be90-2408ab990701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:30.210383 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e22f2-4f55-41f0-be90-2408ab990701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:30.212161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e22f2-4f55-41f0-be90-2408ab990701 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:30.223124 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1393/2772] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:27:30.228252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689eb12-74a7-4566-965b-737419827134 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:30.238748 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689eb12-74a7-4566-965b-737419827134 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:30.243742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa116562-9251-4036-bb34-365c792da8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:30.244746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa116562-9251-4036-bb34-365c792da8c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:30.244746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa116562-9251-4036-bb34-365c792da8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:30.244927 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa116562-9251-4036-bb34-365c792da8c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:30.245712 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa116562-9251-4036-bb34-365c792da8c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:30.247586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa116562-9251-4036-bb34-365c792da8c6 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:30.264069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1380/2773] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:30.264412 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e22f2-4f55-41f0-be90-2408ab990701 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:30.278239 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e22f2-4f55-41f0-be90-2408ab990701 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:30.283037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17345271-7216-495c-bed1-6b0c7b1439c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:30.283728 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17345271-7216-495c-bed1-6b0c7b1439c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:30.284017 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17345271-7216-495c-bed1-6b0c7b1439c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:30.284297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17345271-7216-495c-bed1-6b0c7b1439c2 admin admin] PATH_INFO: `/v3/projects/9e5a79f26d3345749c4b3cbd46653f0d/users/a5eb99a9cad24d34bce1d0ecde276793/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:30.312616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17345271-7216-495c-bed1-6b0c7b1439c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5a79f26d3345749c4b3cbd46653f0d, user_id=a5eb99a9cad24d34bce1d0ecde276793, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:30.315443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17345271-7216-495c-bed1-6b0c7b1439c2 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:30.354352 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1394/2774] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:30 2022] PUT /identity/v3/projects/9e5a79f26d3345749c4b3cbd46653f0d/users/a5eb99a9cad24d34bce1d0ecde276793/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:30.367069 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d2fd3-9cb4-40de-b6a0-adbb9c8d9af6 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:30.367385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d2fd3-9cb4-40de-b6a0-adbb9c8d9af6 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:30.367647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d2fd3-9cb4-40de-b6a0-adbb9c8d9af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:30.406727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:30.419409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-370d2fd3-9cb4-40de-b6a0-adbb9c8d9af6 None None] MFA Rules not processed for user `a5eb99a9cad24d34bce1d0ecde276793`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:30.421134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-370d2fd3-9cb4-40de-b6a0-adbb9c8d9af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:30.528733 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1381/2775] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:30.539827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17345271-7216-495c-bed1-6b0c7b1439c2 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:30.542570 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-17345271-7216-495c-bed1-6b0c7b1439c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:30.615938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17345271-7216-495c-bed1-6b0c7b1439c2 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:30.620886 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76109c40-c3f7-4e0b-ab4b-71284f66b7cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:30.621768 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76109c40-c3f7-4e0b-ab4b-71284f66b7cc service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:30.622044 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76109c40-c3f7-4e0b-ab4b-71284f66b7cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:30.622316 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76109c40-c3f7-4e0b-ab4b-71284f66b7cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:30.631115 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76109c40-c3f7-4e0b-ab4b-71284f66b7cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:30.632598 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76109c40-c3f7-4e0b-ab4b-71284f66b7cc service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:30.642589 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1395/2776] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:27:30.663645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d2fd3-9cb4-40de-b6a0-adbb9c8d9af6 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:30.734491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d2fd3-9cb4-40de-b6a0-adbb9c8d9af6 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:30.739045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:30.739704 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:30.739982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:30.740280 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] PATH_INFO: `/v3/users/a5eb99a9cad24d34bce1d0ecde276793` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:30.741412 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5eb99a9cad24d34bce1d0ecde276793)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:30.743858 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:30.794378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5eb99a9cad24d34bce1d0ecde276793'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:30.809574 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5eb99a9cad24d34bce1d0ecde276793'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:30.810215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5eb99a9cad24d34bce1d0ecde276793'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:30.810807 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5eb99a9cad24d34bce1d0ecde276793'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:30.823933 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5eb99a9cad24d34bce1d0ecde276793'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:30.837193 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1382/2777] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:30 2022] DELETE /identity/v3/users/a5eb99a9cad24d34bce1d0ecde276793 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:30.841596 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f71e34-0b9e-4f65-92db-f788dd4d1f76 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:30.842101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f71e34-0b9e-4f65-92db-f788dd4d1f76 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:30.842937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f71e34-0b9e-4f65-92db-f788dd4d1f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:30.865782 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-28f71e34-0b9e-4f65-92db-f788dd4d1f76 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:30.867031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-28f71e34-0b9e-4f65-92db-f788dd4d1f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:30.942588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1396/2778] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:27:30.953327 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:30.954802 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:30.979192 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:31.019864 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c397aad-7d93-43c2-a6c1-ba4d142914e5 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:31.023533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca5a14-45ae-4f49-ba98-fcf2a6e396a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:31.024181 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca5a14-45ae-4f49-ba98-fcf2a6e396a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:31.024460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca5a14-45ae-4f49-ba98-fcf2a6e396a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:31.025367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca5a14-45ae-4f49-ba98-fcf2a6e396a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:31.035853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca5a14-45ae-4f49-ba98-fcf2a6e396a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:31.037331 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca5a14-45ae-4f49-ba98-fcf2a6e396a4 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:31.048073 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1383/2779] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:27:31.369784 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f71e34-0b9e-4f65-92db-f788dd4d1f76 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:31.371978 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-28f71e34-0b9e-4f65-92db-f788dd4d1f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:31.393214 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f71e34-0b9e-4f65-92db-f788dd4d1f76 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:31.408325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:31.409010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:31.409287 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:31.409568 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8 admin admin] PATH_INFO: `/v3/projects/9e5a79f26d3345749c4b3cbd46653f0d` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:31.410805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e5a79f26d3345749c4b3cbd46653f0d)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:31.412786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:31.416924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:31.472497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e5a79f26d3345749c4b3cbd46653f0d', 'request_id': 'req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:31.480064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e5a79f26d3345749c4b3cbd46653f0d', 'request_id': 'req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:31.485580 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e5a79f26d3345749c4b3cbd46653f0d', 'request_id': 'req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:31.497971 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8 admin admin] The token cache is being invalidate because project 9e5a79f26d3345749c4b3cbd46653f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:31.498269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:31.498918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-569bd750-c32c-4b61-8e51-0f0bebfb9ff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:31.500450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1397/2780] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:31 2022] DELETE /identity/v3/projects/9e5a79f26d3345749c4b3cbd46653f0d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:31.505347 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901ee53-15cb-422c-8c9c-8313aa5742bb None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:31.505629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901ee53-15cb-422c-8c9c-8313aa5742bb None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:31.505905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901ee53-15cb-422c-8c9c-8313aa5742bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:31.526085 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-0901ee53-15cb-422c-8c9c-8313aa5742bb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:31.527290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0901ee53-15cb-422c-8c9c-8313aa5742bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:31.587900 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1384/2781] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:31.595757 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65ef22-0a90-49e3-8d09-afae5798ff3e None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:31.596065 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65ef22-0a90-49e3-8d09-afae5798ff3e None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:31.596371 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65ef22-0a90-49e3-8d09-afae5798ff3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:31.616095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ea65ef22-0a90-49e3-8d09-afae5798ff3e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:31.617303 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ea65ef22-0a90-49e3-8d09-afae5798ff3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:31.626284 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1398/2782] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:31.630537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901ee53-15cb-422c-8c9c-8313aa5742bb None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:31.641458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901ee53-15cb-422c-8c9c-8313aa5742bb None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:31.646019 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ae8e9-1217-498c-829c-8fd843167b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:31.646612 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ae8e9-1217-498c-829c-8fd843167b42 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:31.646905 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ae8e9-1217-498c-829c-8fd843167b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:31.647153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ae8e9-1217-498c-829c-8fd843167b42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:31.647786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ae8e9-1217-498c-829c-8fd843167b42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:27:31.648122 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ae8e9-1217-498c-829c-8fd843167b42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:31.649829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ae8e9-1217-498c-829c-8fd843167b42 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:31.662147 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1385/2783] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:27:31.667063 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65ef22-0a90-49e3-8d09-afae5798ff3e None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:31.676390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65ef22-0a90-49e3-8d09-afae5798ff3e None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:31.682030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fbc33-c736-4b51-96ac-45e0bd82e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:31.682672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fbc33-c736-4b51-96ac-45e0bd82e071 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:31.682986 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fbc33-c736-4b51-96ac-45e0bd82e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:31.683327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fbc33-c736-4b51-96ac-45e0bd82e071 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:31.684089 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fbc33-c736-4b51-96ac-45e0bd82e071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:31.686290 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fbc33-c736-4b51-96ac-45e0bd82e071 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:31.716890 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1399/2784] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:31 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:31.721008 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ae8e9-1217-498c-829c-8fd843167b42 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:31.731644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ae8e9-1217-498c-829c-8fd843167b42 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:31.790090 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1de10b-0fc8-4e3a-a360-edf0731282f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:31.791295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1de10b-0fc8-4e3a-a360-edf0731282f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:31.791579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1de10b-0fc8-4e3a-a360-edf0731282f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:31.791853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1de10b-0fc8-4e3a-a360-edf0731282f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:31.792533 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1de10b-0fc8-4e3a-a360-edf0731282f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:31.794462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1de10b-0fc8-4e3a-a360-edf0731282f1 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:31.815725 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1386/2785] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:31 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:31.819910 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fbc33-c736-4b51-96ac-45e0bd82e071 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:31.830497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fbc33-c736-4b51-96ac-45e0bd82e071 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:31.835181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77c62b-a72c-416d-b7c8-7894c7a7aa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:31.835789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c62b-a72c-416d-b7c8-7894c7a7aa46 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:31.836105 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c62b-a72c-416d-b7c8-7894c7a7aa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:31.836377 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c62b-a72c-416d-b7c8-7894c7a7aa46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:31.837025 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77c62b-a72c-416d-b7c8-7894c7a7aa46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:31.838920 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77c62b-a72c-416d-b7c8-7894c7a7aa46 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:31.849142 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1400/2786] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:31.853974 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1de10b-0fc8-4e3a-a360-edf0731282f1 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:31.865179 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1de10b-0fc8-4e3a-a360-edf0731282f1 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:31.870565 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae0d61-2e58-4e92-80ce-1ae8674d394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:31.871177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae0d61-2e58-4e92-80ce-1ae8674d394f admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:31.871710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae0d61-2e58-4e92-80ce-1ae8674d394f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:31.872015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae0d61-2e58-4e92-80ce-1ae8674d394f admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:31.872765 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae0d61-2e58-4e92-80ce-1ae8674d394f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:31.874367 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae0d61-2e58-4e92-80ce-1ae8674d394f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:31.886079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1387/2787] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:31.892837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77c62b-a72c-416d-b7c8-7894c7a7aa46 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:31.905201 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77c62b-a72c-416d-b7c8-7894c7a7aa46 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:31.909880 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af4807-8e48-4ae4-bec2-510747d004c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:31.910533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af4807-8e48-4ae4-bec2-510747d004c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:31.910827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af4807-8e48-4ae4-bec2-510747d004c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:31.911082 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af4807-8e48-4ae4-bec2-510747d004c2 admin admin] PATH_INFO: `/v3/projects/dc89a0cc2ae04dcb9c8f5cc48628f2fb/users/be3f85dcb16a4f32bc079e0703d819d2/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:31.926232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af4807-8e48-4ae4-bec2-510747d004c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc89a0cc2ae04dcb9c8f5cc48628f2fb, user_id=be3f85dcb16a4f32bc079e0703d819d2, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:31.928925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af4807-8e48-4ae4-bec2-510747d004c2 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:31.943143 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1401/2788] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:31 2022] PUT /identity/v3/projects/dc89a0cc2ae04dcb9c8f5cc48628f2fb/users/be3f85dcb16a4f32bc079e0703d819d2/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:31.953950 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4704c4-aae7-465b-b01d-80e495c57c84 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:31.954508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4704c4-aae7-465b-b01d-80e495c57c84 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:31.954508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4704c4-aae7-465b-b01d-80e495c57c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:31.982359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-db4704c4-aae7-465b-b01d-80e495c57c84 None None] MFA Rules not processed for user `be3f85dcb16a4f32bc079e0703d819d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:31.983575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-db4704c4-aae7-465b-b01d-80e495c57c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:32.001266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:32.049590 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1388/2789] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:32.061465 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af4807-8e48-4ae4-bec2-510747d004c2 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:32.063136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-76af4807-8e48-4ae4-bec2-510747d004c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:32.120077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af4807-8e48-4ae4-bec2-510747d004c2 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:32.123793 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397bfbf-b33e-4a72-90bb-fc507cadc7a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:32.124423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397bfbf-b33e-4a72-90bb-fc507cadc7a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:32.124699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397bfbf-b33e-4a72-90bb-fc507cadc7a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:32.124960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397bfbf-b33e-4a72-90bb-fc507cadc7a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:32.133181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397bfbf-b33e-4a72-90bb-fc507cadc7a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:32.134712 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397bfbf-b33e-4a72-90bb-fc507cadc7a2 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:32.143328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1402/2790] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:42.046755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=19 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:42.139510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:44.128368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4704c4-aae7-465b-b01d-80e495c57c84 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:44.129918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-db4704c4-aae7-465b-b01d-80e495c57c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:44.203740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4704c4-aae7-465b-b01d-80e495c57c84 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:44.208397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452aad2-f5af-4b97-b134-c468e42f8e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:44.209075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452aad2-f5af-4b97-b134-c468e42f8e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:44.209293 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452aad2-f5af-4b97-b134-c468e42f8e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:44.209559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452aad2-f5af-4b97-b134-c468e42f8e56 admin admin] PATH_INFO: `/v3/users/4fe36a87392c40379ac160ebaa3cd84c` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:44.210669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452aad2-f5af-4b97-b134-c468e42f8e56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fe36a87392c40379ac160ebaa3cd84c)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:44.212633 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452aad2-f5af-4b97-b134-c468e42f8e56 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:44.259207 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3452aad2-f5af-4b97-b134-c468e42f8e56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fe36a87392c40379ac160ebaa3cd84c'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:44.267378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3452aad2-f5af-4b97-b134-c468e42f8e56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fe36a87392c40379ac160ebaa3cd84c'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:44.268097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3452aad2-f5af-4b97-b134-c468e42f8e56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fe36a87392c40379ac160ebaa3cd84c'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:44.268710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3452aad2-f5af-4b97-b134-c468e42f8e56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fe36a87392c40379ac160ebaa3cd84c'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:44.280959 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-3452aad2-f5af-4b97-b134-c468e42f8e56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fe36a87392c40379ac160ebaa3cd84c'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:44.293368 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1389/2791] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:44 2022] DELETE /identity/v3/users/4fe36a87392c40379ac160ebaa3cd84c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:44.466173 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397bfbf-b33e-4a72-90bb-fc507cadc7a2 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:44.479486 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397bfbf-b33e-4a72-90bb-fc507cadc7a2 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:44.542370 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18c4f3-3858-43e1-a085-bb4764d04aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:44.543229 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18c4f3-3858-43e1-a085-bb4764d04aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:44.543512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18c4f3-3858-43e1-a085-bb4764d04aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:44.543788 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18c4f3-3858-43e1-a085-bb4764d04aaf admin admin] PATH_INFO: `/v3/projects/f5ddbf041c444253aea027770188fbaf` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:44.544941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18c4f3-3858-43e1-a085-bb4764d04aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ddbf041c444253aea027770188fbaf)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:44.546883 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18c4f3-3858-43e1-a085-bb4764d04aaf admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:44.596981 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9a18c4f3-3858-43e1-a085-bb4764d04aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ddbf041c444253aea027770188fbaf', 'request_id': 'req-9a18c4f3-3858-43e1-a085-bb4764d04aaf'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:44.604640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9a18c4f3-3858-43e1-a085-bb4764d04aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ddbf041c444253aea027770188fbaf', 'request_id': 'req-9a18c4f3-3858-43e1-a085-bb4764d04aaf'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:44.610843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9a18c4f3-3858-43e1-a085-bb4764d04aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ddbf041c444253aea027770188fbaf', 'request_id': 'req-9a18c4f3-3858-43e1-a085-bb4764d04aaf'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:44.621878 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9a18c4f3-3858-43e1-a085-bb4764d04aaf admin admin] The token cache is being invalidate because project f5ddbf041c444253aea027770188fbaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:44.622164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9a18c4f3-3858-43e1-a085-bb4764d04aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:44.622806 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-9a18c4f3-3858-43e1-a085-bb4764d04aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:44.624571 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1403/2792] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:44 2022] DELETE /identity/v3/projects/f5ddbf041c444253aea027770188fbaf => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:44.629491 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b26d80-0540-43d3-8a4f-f07f3edb9c79 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:44.629782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b26d80-0540-43d3-8a4f-f07f3edb9c79 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:44.630060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b26d80-0540-43d3-8a4f-f07f3edb9c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:44.650400 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-f0b26d80-0540-43d3-8a4f-f07f3edb9c79 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:44.651247 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f0b26d80-0540-43d3-8a4f-f07f3edb9c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:44.712257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1390/2793] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:44.720786 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45024c7-5cea-49d4-b26c-28bfe9ee47bb None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:44.721064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45024c7-5cea-49d4-b26c-28bfe9ee47bb None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:44.721336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45024c7-5cea-49d4-b26c-28bfe9ee47bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:44.741188 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-e45024c7-5cea-49d4-b26c-28bfe9ee47bb None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:44.742395 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e45024c7-5cea-49d4-b26c-28bfe9ee47bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:44.751027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1404/2794] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:44.755290 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b26d80-0540-43d3-8a4f-f07f3edb9c79 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:44.765511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b26d80-0540-43d3-8a4f-f07f3edb9c79 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:44.770062 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d95f78-c094-45d4-a61d-1dd866781dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:44.770650 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d95f78-c094-45d4-a61d-1dd866781dde admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:44.770921 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d95f78-c094-45d4-a61d-1dd866781dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:44.771183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d95f78-c094-45d4-a61d-1dd866781dde admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:44.771833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d95f78-c094-45d4-a61d-1dd866781dde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:27:44.772188 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d95f78-c094-45d4-a61d-1dd866781dde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:44.773758 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d95f78-c094-45d4-a61d-1dd866781dde admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:44.785742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1391/2795] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:27:44.790132 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45024c7-5cea-49d4-b26c-28bfe9ee47bb None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:44.799903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45024c7-5cea-49d4-b26c-28bfe9ee47bb None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:44.804578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffe9b9-f966-45ae-a4b5-ec7f383b17b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:44.805170 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffe9b9-f966-45ae-a4b5-ec7f383b17b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:44.805448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffe9b9-f966-45ae-a4b5-ec7f383b17b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:44.805736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffe9b9-f966-45ae-a4b5-ec7f383b17b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:44.806344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ffe9b9-f966-45ae-a4b5-ec7f383b17b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:44.808302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ffe9b9-f966-45ae-a4b5-ec7f383b17b0 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:44.836382 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1405/2796] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:44 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:44.841150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d95f78-c094-45d4-a61d-1dd866781dde admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:44.850975 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d95f78-c094-45d4-a61d-1dd866781dde admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:44.854782 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:44.910976 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b660a-c6b4-4685-b286-25130fdff0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:44.911579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b660a-c6b4-4685-b286-25130fdff0d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:44.911859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b660a-c6b4-4685-b286-25130fdff0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:44.912344 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b660a-c6b4-4685-b286-25130fdff0d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:44.912988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b660a-c6b4-4685-b286-25130fdff0d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:44.914970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b660a-c6b4-4685-b286-25130fdff0d8 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:44.936535 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1392/2797] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:44 2022] POST /identity/v3/users => generated 303 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:44.940805 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffe9b9-f966-45ae-a4b5-ec7f383b17b0 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:44.951702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffe9b9-f966-45ae-a4b5-ec7f383b17b0 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:44.956362 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb4384-31ab-4f38-9595-38f175d14dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:44.956948 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb4384-31ab-4f38-9595-38f175d14dca admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:44.957220 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb4384-31ab-4f38-9595-38f175d14dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:44.957485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb4384-31ab-4f38-9595-38f175d14dca admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:44.958118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb4384-31ab-4f38-9595-38f175d14dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:44.959689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb4384-31ab-4f38-9595-38f175d14dca admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:44.969491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1406/2798] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:44.973407 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b660a-c6b4-4685-b286-25130fdff0d8 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:44.983979 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b660a-c6b4-4685-b286-25130fdff0d8 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:44.990361 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abf78b-aab4-4ee6-a611-afae7af81296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:44.991039 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abf78b-aab4-4ee6-a611-afae7af81296 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:44.991357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abf78b-aab4-4ee6-a611-afae7af81296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:44.991673 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abf78b-aab4-4ee6-a611-afae7af81296 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:44.992401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abf78b-aab4-4ee6-a611-afae7af81296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:44.994123 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abf78b-aab4-4ee6-a611-afae7af81296 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:45.006343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1393/2799] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:45.010736 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb4384-31ab-4f38-9595-38f175d14dca admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:45.022776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb4384-31ab-4f38-9595-38f175d14dca admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:45.027708 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8ad07-51ac-4832-a2e2-6a20b4fa77c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:45.028399 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8ad07-51ac-4832-a2e2-6a20b4fa77c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:45.028681 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8ad07-51ac-4832-a2e2-6a20b4fa77c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:45.028956 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8ad07-51ac-4832-a2e2-6a20b4fa77c4 admin admin] PATH_INFO: `/v3/projects/c846a33965a54e95bd38c52cfff5d8d2/users/9b369af6665e49caa18c1a65db76de3b/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:45.043673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8ad07-51ac-4832-a2e2-6a20b4fa77c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c846a33965a54e95bd38c52cfff5d8d2, user_id=9b369af6665e49caa18c1a65db76de3b, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:45.046396 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8ad07-51ac-4832-a2e2-6a20b4fa77c4 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:45.061617 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1407/2800] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:45 2022] PUT /identity/v3/projects/c846a33965a54e95bd38c52cfff5d8d2/users/9b369af6665e49caa18c1a65db76de3b/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:45.067047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f235e-1149-4f71-9f11-e3c7449d1392 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:45.067333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f235e-1149-4f71-9f11-e3c7449d1392 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:45.067613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f235e-1149-4f71-9f11-e3c7449d1392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:45.087086 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-921f235e-1149-4f71-9f11-e3c7449d1392 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:45.088341 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-921f235e-1149-4f71-9f11-e3c7449d1392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:45.146801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1394/2801] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:45.156659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8ad07-51ac-4832-a2e2-6a20b4fa77c4 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:45.158336 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d5f8ad07-51ac-4832-a2e2-6a20b4fa77c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:45.189826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:45.218097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8ad07-51ac-4832-a2e2-6a20b4fa77c4 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:45.221831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442eaeac-7e3e-4fd7-bd95-9e99b003f7e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:45.222431 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442eaeac-7e3e-4fd7-bd95-9e99b003f7e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:45.222702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442eaeac-7e3e-4fd7-bd95-9e99b003f7e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:45.222966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442eaeac-7e3e-4fd7-bd95-9e99b003f7e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:45.232423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442eaeac-7e3e-4fd7-bd95-9e99b003f7e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:45.233837 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442eaeac-7e3e-4fd7-bd95-9e99b003f7e1 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:45.243026 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1408/2802] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:45.888718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e60055-958e-4803-b791-3de8f937af94 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:45.889001 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e60055-958e-4803-b791-3de8f937af94 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:45.889269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e60055-958e-4803-b791-3de8f937af94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:45.920389 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-91e60055-958e-4803-b791-3de8f937af94 None None] MFA Rules not processed for user `9b369af6665e49caa18c1a65db76de3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:45.921611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-91e60055-958e-4803-b791-3de8f937af94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:45.995503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1395/2803] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:45 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:46.000802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442eaeac-7e3e-4fd7-bd95-9e99b003f7e1 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:46.010706 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442eaeac-7e3e-4fd7-bd95-9e99b003f7e1 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:46.016077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77ee4b-b6a8-4c88-a5a5-c183a447049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:46.016684 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77ee4b-b6a8-4c88-a5a5-c183a447049d admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:46.016961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77ee4b-b6a8-4c88-a5a5-c183a447049d admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:46.017233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77ee4b-b6a8-4c88-a5a5-c183a447049d admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:46.019148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77ee4b-b6a8-4c88-a5a5-c183a447049d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:46.019830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77ee4b-b6a8-4c88-a5a5-c183a447049d admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:46.057918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1409/2804] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:45 2022] POST /identity/v3/projects => generated 401 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:46.061952 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e60055-958e-4803-b791-3de8f937af94 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:46.076851 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e60055-958e-4803-b791-3de8f937af94 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:46.091739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:46.145884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e30a2-bd17-4d32-b440-d3d27e301ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:46.146495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e30a2-bd17-4d32-b440-d3d27e301ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:46.146769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e30a2-bd17-4d32-b440-d3d27e301ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:46.147037 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e30a2-bd17-4d32-b440-d3d27e301ee7 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:46.147669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e30a2-bd17-4d32-b440-d3d27e301ee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:46.149604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e30a2-bd17-4d32-b440-d3d27e301ee7 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:46.175962 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1396/2805] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:46 2022] POST /identity/v3/users => generated 307 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:46.180094 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77ee4b-b6a8-4c88-a5a5-c183a447049d admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:46.191339 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77ee4b-b6a8-4c88-a5a5-c183a447049d admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:46.196334 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24796c-9f50-4680-9803-bdce1469a4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:46.197083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24796c-9f50-4680-9803-bdce1469a4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:46.197425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24796c-9f50-4680-9803-bdce1469a4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:46.197699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24796c-9f50-4680-9803-bdce1469a4a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:46.198348 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24796c-9f50-4680-9803-bdce1469a4a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:46.200620 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24796c-9f50-4680-9803-bdce1469a4a3 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:46.214581 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1410/2806] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:46.219665 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e30a2-bd17-4d32-b440-d3d27e301ee7 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:46.231605 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e30a2-bd17-4d32-b440-d3d27e301ee7 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:46.236956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df86aa7-f3a6-4d86-8b89-a5c0e0c1b768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:46.237937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df86aa7-f3a6-4d86-8b89-a5c0e0c1b768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:46.239818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df86aa7-f3a6-4d86-8b89-a5c0e0c1b768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:46.239818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df86aa7-f3a6-4d86-8b89-a5c0e0c1b768 admin admin] PATH_INFO: `/v3/projects/798283d1f1ba46f186cf2fa6e89a9411/users/2276abac583043539938d654c57a1836/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:46.256365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df86aa7-f3a6-4d86-8b89-a5c0e0c1b768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798283d1f1ba46f186cf2fa6e89a9411, user_id=2276abac583043539938d654c57a1836, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:46.259072 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df86aa7-f3a6-4d86-8b89-a5c0e0c1b768 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:46.288428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1397/2807] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:46 2022] PUT /identity/v3/projects/798283d1f1ba46f186cf2fa6e89a9411/users/2276abac583043539938d654c57a1836/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:46.292332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24796c-9f50-4680-9803-bdce1469a4a3 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:46.305696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24796c-9f50-4680-9803-bdce1469a4a3 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:46.370415 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb5c8e-c539-474b-84be-5015af03f69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:46.372119 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb5c8e-c539-474b-84be-5015af03f69e admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:46.372505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb5c8e-c539-474b-84be-5015af03f69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:46.372704 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb5c8e-c539-474b-84be-5015af03f69e admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:46.373480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb5c8e-c539-474b-84be-5015af03f69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:46.375283 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb5c8e-c539-474b-84be-5015af03f69e admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:46.385809 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1411/2808] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:46.391177 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df86aa7-f3a6-4d86-8b89-a5c0e0c1b768 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:46.401375 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df86aa7-f3a6-4d86-8b89-a5c0e0c1b768 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:46.405919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fba5d8-9431-4d62-b0e3-885a39cf3c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:46.406564 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fba5d8-9431-4d62-b0e3-885a39cf3c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:46.406833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fba5d8-9431-4d62-b0e3-885a39cf3c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:46.407096 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fba5d8-9431-4d62-b0e3-885a39cf3c71 admin admin] PATH_INFO: `/v3/projects/798283d1f1ba46f186cf2fa6e89a9411/users/2276abac583043539938d654c57a1836/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:46.409017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fba5d8-9431-4d62-b0e3-885a39cf3c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798283d1f1ba46f186cf2fa6e89a9411, user_id=2276abac583043539938d654c57a1836, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:46.411613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fba5d8-9431-4d62-b0e3-885a39cf3c71 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:46.421753 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1398/2809] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:46 2022] PUT /identity/v3/projects/798283d1f1ba46f186cf2fa6e89a9411/users/2276abac583043539938d654c57a1836/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:47.079714 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97b4d4-d413-45f6-8c1b-7c38414ba412 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:47.080291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97b4d4-d413-45f6-8c1b-7c38414ba412 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:47.080291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97b4d4-d413-45f6-8c1b-7c38414ba412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:47.110147 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-4e97b4d4-d413-45f6-8c1b-7c38414ba412 None None] MFA Rules not processed for user `2276abac583043539938d654c57a1836`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:47.111684 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-4e97b4d4-d413-45f6-8c1b-7c38414ba412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:47.184501 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1412/2810] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:47 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:48.234956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fba5d8-9431-4d62-b0e3-885a39cf3c71 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:48.302859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fba5d8-9431-4d62-b0e3-885a39cf3c71 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:48.307687 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:48.308357 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:48.308638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:48.308982 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] PATH_INFO: `/v3/users/9b369af6665e49caa18c1a65db76de3b` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:48.310337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b369af6665e49caa18c1a65db76de3b)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:48.312319 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:48.359599 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b369af6665e49caa18c1a65db76de3b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.367273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b369af6665e49caa18c1a65db76de3b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.368129 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b369af6665e49caa18c1a65db76de3b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.368673 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b369af6665e49caa18c1a65db76de3b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.380018 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:48.381212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b369af6665e49caa18c1a65db76de3b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.395427 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1399/2811] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:48 2022] DELETE /identity/v3/users/9b369af6665e49caa18c1a65db76de3b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:48.580932 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97b4d4-d413-45f6-8c1b-7c38414ba412 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:48.592442 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97b4d4-d413-45f6-8c1b-7c38414ba412 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:48.611198 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:48.661412 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:48.662040 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:48.662317 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:48.662585 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] PATH_INFO: `/v3/projects/c846a33965a54e95bd38c52cfff5d8d2` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:48.663698 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c846a33965a54e95bd38c52cfff5d8d2)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:48.665636 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:48.725448 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c846a33965a54e95bd38c52cfff5d8d2', 'request_id': 'req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.733853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c846a33965a54e95bd38c52cfff5d8d2', 'request_id': 'req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.739627 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c846a33965a54e95bd38c52cfff5d8d2', 'request_id': 'req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.751771 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] The token cache is being invalidate because project c846a33965a54e95bd38c52cfff5d8d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:48.752080 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.752838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.754330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1413/2812] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:48 2022] DELETE /identity/v3/projects/c846a33965a54e95bd38c52cfff5d8d2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:48.758484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:48.760023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:48.774709 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909608ef-623e-4e93-aead-3b7274f71f7f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:48.837350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f1a15-6f07-4d3a-b092-93abec6ef01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:48.837993 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f1a15-6f07-4d3a-b092-93abec6ef01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:48.838268 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f1a15-6f07-4d3a-b092-93abec6ef01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:48.838540 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f1a15-6f07-4d3a-b092-93abec6ef01b admin admin] PATH_INFO: `/v3/users/2276abac583043539938d654c57a1836` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:48.839677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f1a15-6f07-4d3a-b092-93abec6ef01b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2276abac583043539938d654c57a1836)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:48.842312 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f1a15-6f07-4d3a-b092-93abec6ef01b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:48.887022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-563f1a15-6f07-4d3a-b092-93abec6ef01b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2276abac583043539938d654c57a1836'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.894902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-563f1a15-6f07-4d3a-b092-93abec6ef01b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2276abac583043539938d654c57a1836'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.895502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-563f1a15-6f07-4d3a-b092-93abec6ef01b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2276abac583043539938d654c57a1836'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.896144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-563f1a15-6f07-4d3a-b092-93abec6ef01b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2276abac583043539938d654c57a1836'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.908676 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-563f1a15-6f07-4d3a-b092-93abec6ef01b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2276abac583043539938d654c57a1836'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:48.921158 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1400/2813] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:48 2022] DELETE /identity/v3/users/2276abac583043539938d654c57a1836 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:49.101692 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:49.103113 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:49.117086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf537852-b174-4baa-ab5e-d60b8f2d5ed0 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:49.175576 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a624d7-7151-4b22-8cf3-159898282101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:49.176110 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a624d7-7151-4b22-8cf3-159898282101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:49.176384 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a624d7-7151-4b22-8cf3-159898282101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:49.176651 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a624d7-7151-4b22-8cf3-159898282101 admin admin] PATH_INFO: `/v3/projects/798283d1f1ba46f186cf2fa6e89a9411` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:49.177777 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a624d7-7151-4b22-8cf3-159898282101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=798283d1f1ba46f186cf2fa6e89a9411)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:49.179668 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a624d7-7151-4b22-8cf3-159898282101 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:49.224886 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-63a624d7-7151-4b22-8cf3-159898282101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798283d1f1ba46f186cf2fa6e89a9411', 'request_id': 'req-63a624d7-7151-4b22-8cf3-159898282101'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:49.226497 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:49.232943 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-63a624d7-7151-4b22-8cf3-159898282101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798283d1f1ba46f186cf2fa6e89a9411', 'request_id': 'req-63a624d7-7151-4b22-8cf3-159898282101'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:49.239551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-63a624d7-7151-4b22-8cf3-159898282101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798283d1f1ba46f186cf2fa6e89a9411', 'request_id': 'req-63a624d7-7151-4b22-8cf3-159898282101'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:49.251347 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-63a624d7-7151-4b22-8cf3-159898282101 admin admin] The token cache is being invalidate because project 798283d1f1ba46f186cf2fa6e89a9411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:49.251634 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-63a624d7-7151-4b22-8cf3-159898282101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:49.252291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-63a624d7-7151-4b22-8cf3-159898282101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:49.253758 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1414/2814] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:49 2022] DELETE /identity/v3/projects/798283d1f1ba46f186cf2fa6e89a9411 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:49.259527 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4f894-8862-4f1e-b5c4-e4ed4edabdee None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:49.260098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4f894-8862-4f1e-b5c4-e4ed4edabdee None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:49.260098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4f894-8862-4f1e-b5c4-e4ed4edabdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:49.280269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-66b4f894-8862-4f1e-b5c4-e4ed4edabdee None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:49.281465 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-66b4f894-8862-4f1e-b5c4-e4ed4edabdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:49.340835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1401/2815] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:49.348587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc5125-af62-4160-ace1-ec0e4838a7bf None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:49.348869 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc5125-af62-4160-ace1-ec0e4838a7bf None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:49.349137 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc5125-af62-4160-ace1-ec0e4838a7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:49.369306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-accc5125-af62-4160-ace1-ec0e4838a7bf None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:49.370496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-accc5125-af62-4160-ace1-ec0e4838a7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:49.378955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1415/2816] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:49.383308 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4f894-8862-4f1e-b5c4-e4ed4edabdee None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:49.392988 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4f894-8862-4f1e-b5c4-e4ed4edabdee None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:49.397551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca6c12-a145-4d38-aa4f-6f839a1fd46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:49.398143 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca6c12-a145-4d38-aa4f-6f839a1fd46b admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:49.398415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca6c12-a145-4d38-aa4f-6f839a1fd46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:49.398680 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca6c12-a145-4d38-aa4f-6f839a1fd46b admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:49.399318 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca6c12-a145-4d38-aa4f-6f839a1fd46b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:27:49.399613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca6c12-a145-4d38-aa4f-6f839a1fd46b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:49.401215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca6c12-a145-4d38-aa4f-6f839a1fd46b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:49.402651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:49.416045 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1402/2817] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:27:49.421183 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc5125-af62-4160-ace1-ec0e4838a7bf None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:49.430985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc5125-af62-4160-ace1-ec0e4838a7bf None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:49.435839 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede2329-5d94-443c-a828-c3f1bba038a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:49.436473 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede2329-5d94-443c-a828-c3f1bba038a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:49.436749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede2329-5d94-443c-a828-c3f1bba038a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:49.437016 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede2329-5d94-443c-a828-c3f1bba038a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:49.437673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede2329-5d94-443c-a828-c3f1bba038a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:49.439560 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede2329-5d94-443c-a828-c3f1bba038a0 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:49.467551 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1416/2818] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:49 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:49.471880 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca6c12-a145-4d38-aa4f-6f839a1fd46b admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:49.482999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca6c12-a145-4d38-aa4f-6f839a1fd46b admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:49.537644 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a5ccc-8c81-4aae-9f89-5f3f5310feaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:49.538240 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a5ccc-8c81-4aae-9f89-5f3f5310feaf admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:49.538514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a5ccc-8c81-4aae-9f89-5f3f5310feaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:49.538781 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a5ccc-8c81-4aae-9f89-5f3f5310feaf admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:49.539398 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a5ccc-8c81-4aae-9f89-5f3f5310feaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:49.541321 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a5ccc-8c81-4aae-9f89-5f3f5310feaf admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:49.563134 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1403/2819] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:49 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:49.567474 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede2329-5d94-443c-a828-c3f1bba038a0 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:49.578409 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede2329-5d94-443c-a828-c3f1bba038a0 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:49.583271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad93c1a-efe9-4c6b-9f2e-a2540a963cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:49.583910 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad93c1a-efe9-4c6b-9f2e-a2540a963cda admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:49.584246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad93c1a-efe9-4c6b-9f2e-a2540a963cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:49.584519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad93c1a-efe9-4c6b-9f2e-a2540a963cda admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:49.585169 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad93c1a-efe9-4c6b-9f2e-a2540a963cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:49.586775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad93c1a-efe9-4c6b-9f2e-a2540a963cda admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:49.597522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1417/2820] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:49.601726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a5ccc-8c81-4aae-9f89-5f3f5310feaf admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:49.612478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a5ccc-8c81-4aae-9f89-5f3f5310feaf admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:49.617157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd740c3-3573-47d5-91fe-8b2ec986d54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:49.617755 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd740c3-3573-47d5-91fe-8b2ec986d54d admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:49.618036 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd740c3-3573-47d5-91fe-8b2ec986d54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:49.618307 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd740c3-3573-47d5-91fe-8b2ec986d54d admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:49.618949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd740c3-3573-47d5-91fe-8b2ec986d54d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:49.620560 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd740c3-3573-47d5-91fe-8b2ec986d54d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:49.631291 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1404/2821] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:49.635550 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad93c1a-efe9-4c6b-9f2e-a2540a963cda admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:49.645624 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad93c1a-efe9-4c6b-9f2e-a2540a963cda admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:49.650246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a364d-3d5a-40b7-82ba-e4090a152d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:49.650894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a364d-3d5a-40b7-82ba-e4090a152d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:49.651172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a364d-3d5a-40b7-82ba-e4090a152d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:49.651441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a364d-3d5a-40b7-82ba-e4090a152d3b admin admin] PATH_INFO: `/v3/projects/7622f99fe4ce4c43b3486c081462bd10/users/11084ab6fb8b433fb1adfa1d6eb0c70d/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:49.666538 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a364d-3d5a-40b7-82ba-e4090a152d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7622f99fe4ce4c43b3486c081462bd10, user_id=11084ab6fb8b433fb1adfa1d6eb0c70d, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:49.669859 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a364d-3d5a-40b7-82ba-e4090a152d3b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:49.683924 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1418/2822] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:49 2022] PUT /identity/v3/projects/7622f99fe4ce4c43b3486c081462bd10/users/11084ab6fb8b433fb1adfa1d6eb0c70d/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:49.695768 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb9704-31de-4f40-83f6-31bc8f4e7ce5 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:49.696091 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb9704-31de-4f40-83f6-31bc8f4e7ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:49.696408 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb9704-31de-4f40-83f6-31bc8f4e7ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:49.727663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8fdb9704-31de-4f40-83f6-31bc8f4e7ce5 None None] MFA Rules not processed for user `11084ab6fb8b433fb1adfa1d6eb0c70d`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:49.728908 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8fdb9704-31de-4f40-83f6-31bc8f4e7ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:49.796062 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1405/2823] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:49.800560 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a364d-3d5a-40b7-82ba-e4090a152d3b admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:49.868891 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a364d-3d5a-40b7-82ba-e4090a152d3b admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:49.873667 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b7c12-d413-441f-8c80-a4e925696068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:49.874297 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b7c12-d413-441f-8c80-a4e925696068 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:49.874579 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b7c12-d413-441f-8c80-a4e925696068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:49.874852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b7c12-d413-441f-8c80-a4e925696068 admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:49.875776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b7c12-d413-441f-8c80-a4e925696068 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:49.877763 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b7c12-d413-441f-8c80-a4e925696068 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:49.909392 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1419/2824] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:49 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:49.910596 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb9704-31de-4f40-83f6-31bc8f4e7ce5 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:49.922637 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb9704-31de-4f40-83f6-31bc8f4e7ce5 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:49.972498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b7c12-d413-441f-8c80-a4e925696068 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:49.974181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ff1b7c12-d413-441f-8c80-a4e925696068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:49.982126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27c640-f2d2-4ed9-a1d2-9c80470eade8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:49.982744 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27c640-f2d2-4ed9-a1d2-9c80470eade8 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:49.983015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27c640-f2d2-4ed9-a1d2-9c80470eade8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:49.983277 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27c640-f2d2-4ed9-a1d2-9c80470eade8 admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:49.983899 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27c640-f2d2-4ed9-a1d2-9c80470eade8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:49.986301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27c640-f2d2-4ed9-a1d2-9c80470eade8 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:49.995363 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:50.009144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1406/2825] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:49 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:50.013057 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27c640-f2d2-4ed9-a1d2-9c80470eade8 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:50.022999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27c640-f2d2-4ed9-a1d2-9c80470eade8 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:50.027865 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf57182-9460-4bf2-9ba4-af29ef19f406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:50.028508 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf57182-9460-4bf2-9ba4-af29ef19f406 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.028778 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf57182-9460-4bf2-9ba4-af29ef19f406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.029136 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf57182-9460-4bf2-9ba4-af29ef19f406 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.029769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf57182-9460-4bf2-9ba4-af29ef19f406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:50.031401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf57182-9460-4bf2-9ba4-af29ef19f406 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:50.041359 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1407/2826] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:50.043306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b7c12-d413-441f-8c80-a4e925696068 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:50.046136 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf57182-9460-4bf2-9ba4-af29ef19f406 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:50.049039 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044a722-74b6-4634-91dc-53f390aa9ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:50.049672 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044a722-74b6-4634-91dc-53f390aa9ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.049952 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044a722-74b6-4634-91dc-53f390aa9ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.050217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044a722-74b6-4634-91dc-53f390aa9ad8 admin admin] PATH_INFO: `/v3/users/be3f85dcb16a4f32bc079e0703d819d2` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.051354 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044a722-74b6-4634-91dc-53f390aa9ad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be3f85dcb16a4f32bc079e0703d819d2)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:50.053305 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044a722-74b6-4634-91dc-53f390aa9ad8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:50.054964 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:50.055691 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf57182-9460-4bf2-9ba4-af29ef19f406 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:50.061944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58e081-5372-4635-a6d0-69c1b71ef5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:50.062749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58e081-5372-4635-a6d0-69c1b71ef5ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.063026 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58e081-5372-4635-a6d0-69c1b71ef5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.063380 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58e081-5372-4635-a6d0-69c1b71ef5ca admin admin] PATH_INFO: `/v3/projects/fba2a5123b9948ed8ad7bc6cf4c5c322/users/fdc1432fa9aa466bb6b2012f52b6f0e0/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.078984 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58e081-5372-4635-a6d0-69c1b71ef5ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba2a5123b9948ed8ad7bc6cf4c5c322, user_id=fdc1432fa9aa466bb6b2012f52b6f0e0, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:50.082208 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58e081-5372-4635-a6d0-69c1b71ef5ca admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:50.097673 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1408/2827] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:50 2022] PUT /identity/v3/projects/fba2a5123b9948ed8ad7bc6cf4c5c322/users/fdc1432fa9aa466bb6b2012f52b6f0e0/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:50.101729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58e081-5372-4635-a6d0-69c1b71ef5ca admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:50.108491 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b044a722-74b6-4634-91dc-53f390aa9ad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be3f85dcb16a4f32bc079e0703d819d2'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.121378 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b044a722-74b6-4634-91dc-53f390aa9ad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be3f85dcb16a4f32bc079e0703d819d2'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.128877 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b044a722-74b6-4634-91dc-53f390aa9ad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be3f85dcb16a4f32bc079e0703d819d2'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.137717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b044a722-74b6-4634-91dc-53f390aa9ad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be3f85dcb16a4f32bc079e0703d819d2'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.138466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-b044a722-74b6-4634-91dc-53f390aa9ad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be3f85dcb16a4f32bc079e0703d819d2'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.145056 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1420/2828] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:49 2022] DELETE /identity/v3/users/be3f85dcb16a4f32bc079e0703d819d2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:50.149504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d74e5-c3bc-4cbf-86bb-a72ba7c2707f None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.149787 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d74e5-c3bc-4cbf-86bb-a72ba7c2707f None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.150067 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d74e5-c3bc-4cbf-86bb-a72ba7c2707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.169460 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58e081-5372-4635-a6d0-69c1b71ef5ca admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:50.170761 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-ca3d74e5-c3bc-4cbf-86bb-a72ba7c2707f None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:50.171989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ca3d74e5-c3bc-4cbf-86bb-a72ba7c2707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:50.231121 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c2768-57aa-4120-ad4d-85c1fc44a4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:50.231648 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c2768-57aa-4120-ad4d-85c1fc44a4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.232017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c2768-57aa-4120-ad4d-85c1fc44a4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.232324 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c2768-57aa-4120-ad4d-85c1fc44a4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.232956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c2768-57aa-4120-ad4d-85c1fc44a4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:50.234525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c2768-57aa-4120-ad4d-85c1fc44a4c5 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:50.234985 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1421/2829] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:27:50.244504 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d74e5-c3bc-4cbf-86bb-a72ba7c2707f None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:50.245855 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1409/2830] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:27:50.246006 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-ca3d74e5-c3bc-4cbf-86bb-a72ba7c2707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:50.250580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c2768-57aa-4120-ad4d-85c1fc44a4c5 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:50.252282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2a5c2768-57aa-4120-ad4d-85c1fc44a4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:50.264936 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c2768-57aa-4120-ad4d-85c1fc44a4c5 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:50.269941 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869396f-499c-45b3-8728-13a199d57120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:50.270566 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869396f-499c-45b3-8728-13a199d57120 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.270835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869396f-499c-45b3-8728-13a199d57120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.271097 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869396f-499c-45b3-8728-13a199d57120 admin admin] PATH_INFO: `/v3/projects/fba2a5123b9948ed8ad7bc6cf4c5c322/users/fdc1432fa9aa466bb6b2012f52b6f0e0/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.273022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869396f-499c-45b3-8728-13a199d57120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba2a5123b9948ed8ad7bc6cf4c5c322, user_id=fdc1432fa9aa466bb6b2012f52b6f0e0, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:50.275657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869396f-499c-45b3-8728-13a199d57120 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:50.286274 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1410/2831] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:50 2022] PUT /identity/v3/projects/fba2a5123b9948ed8ad7bc6cf4c5c322/users/fdc1432fa9aa466bb6b2012f52b6f0e0/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:50.296556 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6f638-dde2-48c7-893f-f39e97f1fac0 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.296844 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6f638-dde2-48c7-893f-f39e97f1fac0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.297232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6f638-dde2-48c7-893f-f39e97f1fac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.306653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d74e5-c3bc-4cbf-86bb-a72ba7c2707f None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:50.327575 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-a4a6f638-dde2-48c7-893f-f39e97f1fac0 None None] MFA Rules not processed for user `fdc1432fa9aa466bb6b2012f52b6f0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:50.328829 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a4a6f638-dde2-48c7-893f-f39e97f1fac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:50.339480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:50.359845 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1b526-8367-4036-a0fd-e4d35dfda777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:50.360492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1b526-8367-4036-a0fd-e4d35dfda777 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.360773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1b526-8367-4036-a0fd-e4d35dfda777 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.361042 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1b526-8367-4036-a0fd-e4d35dfda777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.363634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:50.401641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1411/2832] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:27:50.413839 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6f638-dde2-48c7-893f-f39e97f1fac0 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:50.415262 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-a4a6f638-dde2-48c7-893f-f39e97f1fac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:50.424084 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1b526-8367-4036-a0fd-e4d35dfda777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:50.425533 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1b526-8367-4036-a0fd-e4d35dfda777 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:50.437076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1422/2833] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:27:50.476679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6f638-dde2-48c7-893f-f39e97f1fac0 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:50.480481 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c4a90-1179-4e69-917d-fcb6fabb697b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:50.481814 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c4a90-1179-4e69-917d-fcb6fabb697b service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.482009 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c4a90-1179-4e69-917d-fcb6fabb697b service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.482277 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c4a90-1179-4e69-917d-fcb6fabb697b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.492161 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c4a90-1179-4e69-917d-fcb6fabb697b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:50.493659 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c4a90-1179-4e69-917d-fcb6fabb697b service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:50.505225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1412/2834] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:50.537398 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1b526-8367-4036-a0fd-e4d35dfda777 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:50.548907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1b526-8367-4036-a0fd-e4d35dfda777 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:50.553715 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:50.554353 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.554630 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.554901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] PATH_INFO: `/v3/users/11084ab6fb8b433fb1adfa1d6eb0c70d` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.556101 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11084ab6fb8b433fb1adfa1d6eb0c70d)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:50.558038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:50.614250 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11084ab6fb8b433fb1adfa1d6eb0c70d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.628128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11084ab6fb8b433fb1adfa1d6eb0c70d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.635176 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11084ab6fb8b433fb1adfa1d6eb0c70d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.643327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11084ab6fb8b433fb1adfa1d6eb0c70d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.643976 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11084ab6fb8b433fb1adfa1d6eb0c70d'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.650340 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1423/2835] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:50 2022] DELETE /identity/v3/users/11084ab6fb8b433fb1adfa1d6eb0c70d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:50.654872 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409287a-98a2-482e-9510-c7e43adead5b None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.655153 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409287a-98a2-482e-9510-c7e43adead5b None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.655424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409287a-98a2-482e-9510-c7e43adead5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.691079 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-e409287a-98a2-482e-9510-c7e43adead5b None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:50.692343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e409287a-98a2-482e-9510-c7e43adead5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:50.742246 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:50.743703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:50.775618 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1413/2836] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:27:50.782273 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409287a-98a2-482e-9510-c7e43adead5b None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:50.783743 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-e409287a-98a2-482e-9510-c7e43adead5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:50.811905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:50.816443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d066b-7711-4f90-afe2-873ad11007d8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:50.823783 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:50.824468 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.824800 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.825128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b admin admin] PATH_INFO: `/v3/projects/dc89a0cc2ae04dcb9c8f5cc48628f2fb` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.826492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc89a0cc2ae04dcb9c8f5cc48628f2fb)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:50.828477 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:50.845932 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409287a-98a2-482e-9510-c7e43adead5b None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:50.849587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836203a-b126-4ce4-affd-c86be7ad84c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:50.850188 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836203a-b126-4ce4-affd-c86be7ad84c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.850473 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836203a-b126-4ce4-affd-c86be7ad84c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.850736 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836203a-b126-4ce4-affd-c86be7ad84c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.861770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836203a-b126-4ce4-affd-c86be7ad84c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:50.863203 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836203a-b126-4ce4-affd-c86be7ad84c6 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:50.873503 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1414/2837] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:27:50.878950 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc89a0cc2ae04dcb9c8f5cc48628f2fb', 'request_id': 'req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.886314 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc89a0cc2ae04dcb9c8f5cc48628f2fb', 'request_id': 'req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.892874 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc89a0cc2ae04dcb9c8f5cc48628f2fb', 'request_id': 'req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.905042 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b admin admin] The token cache is being invalidate because project dc89a0cc2ae04dcb9c8f5cc48628f2fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:50.905330 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.906587 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3fdf7b6c-6189-487e-95dd-5d2a9ddb353b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:50.908503 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1424/2838] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:50 2022] DELETE /identity/v3/projects/dc89a0cc2ae04dcb9c8f5cc48628f2fb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:50.914178 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3318f2-c5ce-4c22-8fb6-65b769df64d9 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:50.914616 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3318f2-c5ce-4c22-8fb6-65b769df64d9 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:50.914906 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3318f2-c5ce-4c22-8fb6-65b769df64d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:50.936626 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-dd3318f2-c5ce-4c22-8fb6-65b769df64d9 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:50.937973 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-dd3318f2-c5ce-4c22-8fb6-65b769df64d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:50.952041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=100 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:51.006920 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1415/2839] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:51.014574 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e94ea9-2126-43fe-83a5-7437f0bc590a None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:51.014857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e94ea9-2126-43fe-83a5-7437f0bc590a None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:51.015133 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e94ea9-2126-43fe-83a5-7437f0bc590a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:51.039703 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-36e94ea9-2126-43fe-83a5-7437f0bc590a None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:51.041232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-36e94ea9-2126-43fe-83a5-7437f0bc590a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:51.049938 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1425/2840] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:27:51.054212 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3318f2-c5ce-4c22-8fb6-65b769df64d9 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:51.064295 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3318f2-c5ce-4c22-8fb6-65b769df64d9 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:51.069150 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2eecf-c141-459f-88f6-4681357edfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:51.069752 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2eecf-c141-459f-88f6-4681357edfae admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:51.070007 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2eecf-c141-459f-88f6-4681357edfae admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:51.070278 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2eecf-c141-459f-88f6-4681357edfae admin admin] PATH_INFO: `/v3/domains` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:51.071050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2eecf-c141-459f-88f6-4681357edfae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81646) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:27:51.071364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2eecf-c141-459f-88f6-4681357edfae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:51.072990 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2eecf-c141-459f-88f6-4681357edfae admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:51.085320 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1416/2841] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:27:51.089961 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e94ea9-2126-43fe-83a5-7437f0bc590a None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:51.100464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e94ea9-2126-43fe-83a5-7437f0bc590a None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:51.107235 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6142a-a8bd-44bd-8a7b-b7e50df42fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:51.108049 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6142a-a8bd-44bd-8a7b-b7e50df42fcf admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:51.108389 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6142a-a8bd-44bd-8a7b-b7e50df42fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:51.108665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6142a-a8bd-44bd-8a7b-b7e50df42fcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:51.109428 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6142a-a8bd-44bd-8a7b-b7e50df42fcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:51.111740 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6142a-a8bd-44bd-8a7b-b7e50df42fcf admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:51.144390 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1426/2842] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:51 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:51.145780 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2eecf-c141-459f-88f6-4681357edfae admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:51.157719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2eecf-c141-459f-88f6-4681357edfae admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:51.161034 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6142a-a8bd-44bd-8a7b-b7e50df42fcf admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:51.162739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d3a6142a-a8bd-44bd-8a7b-b7e50df42fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:51.224356 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab581bd6-9e52-427b-8451-ed23218f06fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:51.225062 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab581bd6-9e52-427b-8451-ed23218f06fc admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:51.225415 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab581bd6-9e52-427b-8451-ed23218f06fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:51.225764 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab581bd6-9e52-427b-8451-ed23218f06fc admin admin] PATH_INFO: `/v3/users` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:51.226471 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab581bd6-9e52-427b-8451-ed23218f06fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:51.228483 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab581bd6-9e52-427b-8451-ed23218f06fc admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:51.230739 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6142a-a8bd-44bd-8a7b-b7e50df42fcf admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:51.236161 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:51.236812 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:51.237095 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:51.237364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] PATH_INFO: `/v3/projects/7622f99fe4ce4c43b3486c081462bd10` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:51.238635 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7622f99fe4ce4c43b3486c081462bd10)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:51.240717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:51.253722 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1417/2843] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:51 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:51.257807 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab581bd6-9e52-427b-8451-ed23218f06fc admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:51.267512 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab581bd6-9e52-427b-8451-ed23218f06fc admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:51.272698 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595541f9-a2a3-4734-a40f-ecabbed3524a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:51.273346 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595541f9-a2a3-4734-a40f-ecabbed3524a admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:51.273641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595541f9-a2a3-4734-a40f-ecabbed3524a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:51.273909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595541f9-a2a3-4734-a40f-ecabbed3524a admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:51.274548 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595541f9-a2a3-4734-a40f-ecabbed3524a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:51.276163 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595541f9-a2a3-4734-a40f-ecabbed3524a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:51.287081 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1418/2844] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:27:51.290949 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595541f9-a2a3-4734-a40f-ecabbed3524a admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:51.297628 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7622f99fe4ce4c43b3486c081462bd10', 'request_id': 'req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:51.304501 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595541f9-a2a3-4734-a40f-ecabbed3524a admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:51.306216 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7622f99fe4ce4c43b3486c081462bd10', 'request_id': 'req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:51.310934 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:51.313843 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7622f99fe4ce4c43b3486c081462bd10', 'request_id': 'req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:51.326897 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] The token cache is being invalidate because project 7622f99fe4ce4c43b3486c081462bd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81647) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:51.327193 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:51.328174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:51.329702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1427/2845] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:51 2022] DELETE /identity/v3/projects/7622f99fe4ce4c43b3486c081462bd10 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:51.333662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:51.335464 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:51.368333 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1043611-3cb0-4db2-bf9f-0c58ae7d4760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:51.368931 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043611-3cb0-4db2-bf9f-0c58ae7d4760 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:51.369204 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043611-3cb0-4db2-bf9f-0c58ae7d4760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:51.369469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043611-3cb0-4db2-bf9f-0c58ae7d4760 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:51.370101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1043611-3cb0-4db2-bf9f-0c58ae7d4760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:51.371666 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1043611-3cb0-4db2-bf9f-0c58ae7d4760 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:51.385392 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1419/2846] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:27:51.386638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1043611-3cb0-4db2-bf9f-0c58ae7d4760 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:51.388842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f1043611-3cb0-4db2-bf9f-0c58ae7d4760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:51.401286 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1043611-3cb0-4db2-bf9f-0c58ae7d4760 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:51.402369 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d0148-9bae-4836-b7a1-a6b0d89cbb73 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:51.406753 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e96cb-0dc5-4516-8646-7ca26c5c2d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:51.407386 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e96cb-0dc5-4516-8646-7ca26c5c2d80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:51.407657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e96cb-0dc5-4516-8646-7ca26c5c2d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:51.407765 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:51.407924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e96cb-0dc5-4516-8646-7ca26c5c2d80 admin admin] PATH_INFO: `/v3/projects/212385b1cc584ef4ae751d3ad3781fdd/users/aca1b07c09b746a1941aada2c359b3a8/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:51.408441 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:51.408718 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:51.408989 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] PATH_INFO: `/v3/users/fdc1432fa9aa466bb6b2012f52b6f0e0` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:51.410182 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdc1432fa9aa466bb6b2012f52b6f0e0)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:51.412311 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:51.423480 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e96cb-0dc5-4516-8646-7ca26c5c2d80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=212385b1cc584ef4ae751d3ad3781fdd, user_id=aca1b07c09b746a1941aada2c359b3a8, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:51.426384 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e96cb-0dc5-4516-8646-7ca26c5c2d80 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:51.429486 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:51.443420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1420/2847] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:51 2022] PUT /identity/v3/projects/212385b1cc584ef4ae751d3ad3781fdd/users/aca1b07c09b746a1941aada2c359b3a8/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:51.454946 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b602ea1-a927-40b1-b0d9-88f2f6371509 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:51.455377 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b602ea1-a927-40b1-b0d9-88f2f6371509 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:51.455774 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b602ea1-a927-40b1-b0d9-88f2f6371509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:51.471072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdc1432fa9aa466bb6b2012f52b6f0e0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:51.484815 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdc1432fa9aa466bb6b2012f52b6f0e0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:51.486031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-8b602ea1-a927-40b1-b0d9-88f2f6371509 None None] MFA Rules not processed for user `aca1b07c09b746a1941aada2c359b3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:51.487403 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8b602ea1-a927-40b1-b0d9-88f2f6371509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:51.492037 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdc1432fa9aa466bb6b2012f52b6f0e0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:51.500799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdc1432fa9aa466bb6b2012f52b6f0e0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:51.501762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdc1432fa9aa466bb6b2012f52b6f0e0'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:51.508512 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1428/2848] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:51 2022] DELETE /identity/v3/users/fdc1432fa9aa466bb6b2012f52b6f0e0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:51.560681 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1421/2849] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:27:51.572751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:51.574174 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:51.640222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa2ff6-2236-4a3b-886d-89500e5446c7 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:51.643895 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67d102-c666-41f6-a3fd-bac4ba402f8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:51.644531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67d102-c666-41f6-a3fd-bac4ba402f8e service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:51.644804 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67d102-c666-41f6-a3fd-bac4ba402f8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:51.645067 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67d102-c666-41f6-a3fd-bac4ba402f8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:51.671190 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:51.715351 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67d102-c666-41f6-a3fd-bac4ba402f8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:51.717980 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67d102-c666-41f6-a3fd-bac4ba402f8e service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:51.727136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1429/2850] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:51.803971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b602ea1-a927-40b1-b0d9-88f2f6371509 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:51.806224 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-8b602ea1-a927-40b1-b0d9-88f2f6371509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:51.881067 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b602ea1-a927-40b1-b0d9-88f2f6371509 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:51.944907 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54423bd-e788-447b-bbf2-b227fb9c116f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:51.945769 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54423bd-e788-447b-bbf2-b227fb9c116f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:51.946148 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54423bd-e788-447b-bbf2-b227fb9c116f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:51.946446 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54423bd-e788-447b-bbf2-b227fb9c116f admin admin] PATH_INFO: `/v3/projects/fba2a5123b9948ed8ad7bc6cf4c5c322` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:51.948048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54423bd-e788-447b-bbf2-b227fb9c116f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fba2a5123b9948ed8ad7bc6cf4c5c322)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:51.950093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54423bd-e788-447b-bbf2-b227fb9c116f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:52.006749 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:52.021279 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b54423bd-e788-447b-bbf2-b227fb9c116f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fba2a5123b9948ed8ad7bc6cf4c5c322', 'request_id': 'req-b54423bd-e788-447b-bbf2-b227fb9c116f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:52.029601 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b54423bd-e788-447b-bbf2-b227fb9c116f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fba2a5123b9948ed8ad7bc6cf4c5c322', 'request_id': 'req-b54423bd-e788-447b-bbf2-b227fb9c116f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:52.035859 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b54423bd-e788-447b-bbf2-b227fb9c116f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fba2a5123b9948ed8ad7bc6cf4c5c322', 'request_id': 'req-b54423bd-e788-447b-bbf2-b227fb9c116f'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:52.056696 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b54423bd-e788-447b-bbf2-b227fb9c116f admin admin] The token cache is being invalidate because project fba2a5123b9948ed8ad7bc6cf4c5c322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:52.056987 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b54423bd-e788-447b-bbf2-b227fb9c116f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:52.057741 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-b54423bd-e788-447b-bbf2-b227fb9c116f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:52.059520 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1422/2851] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:51 2022] DELETE /identity/v3/projects/fba2a5123b9948ed8ad7bc6cf4c5c322 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:52.066289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccc0ac-8280-4be9-98e9-b6bdc72041d7 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:52.066775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccc0ac-8280-4be9-98e9-b6bdc72041d7 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:52.067207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccc0ac-8280-4be9-98e9-b6bdc72041d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:52.088631 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-d7ccc0ac-8280-4be9-98e9-b6bdc72041d7 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:52.090696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d7ccc0ac-8280-4be9-98e9-b6bdc72041d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:52.151853 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1430/2852] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:52.159924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5425e0-b0a5-47d7-aa0d-559212ed4598 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:52.160225 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5425e0-b0a5-47d7-aa0d-559212ed4598 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:52.160496 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5425e0-b0a5-47d7-aa0d-559212ed4598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:52.181365 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-5e5425e0-b0a5-47d7-aa0d-559212ed4598 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:52.183364 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-5e5425e0-b0a5-47d7-aa0d-559212ed4598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:52.194202 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1423/2853] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:52.198650 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccc0ac-8280-4be9-98e9-b6bdc72041d7 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:52.209024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccc0ac-8280-4be9-98e9-b6bdc72041d7 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:52.213633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a3ae1-ecfa-428e-b680-7da4e644e464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:52.214232 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a3ae1-ecfa-428e-b680-7da4e644e464 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:52.214502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a3ae1-ecfa-428e-b680-7da4e644e464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:52.214789 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a3ae1-ecfa-428e-b680-7da4e644e464 admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:52.215447 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a3ae1-ecfa-428e-b680-7da4e644e464 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:27:52.215744 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a3ae1-ecfa-428e-b680-7da4e644e464 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:52.217357 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a3ae1-ecfa-428e-b680-7da4e644e464 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:52.228684 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1431/2854] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:27:52.233790 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5425e0-b0a5-47d7-aa0d-559212ed4598 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:52.243160 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5425e0-b0a5-47d7-aa0d-559212ed4598 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:52.248638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac62e5-f4d5-487c-aef0-24503bc840b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:52.249266 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac62e5-f4d5-487c-aef0-24503bc840b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:52.249542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac62e5-f4d5-487c-aef0-24503bc840b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:52.249812 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac62e5-f4d5-487c-aef0-24503bc840b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:52.250458 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac62e5-f4d5-487c-aef0-24503bc840b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:52.252719 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac62e5-f4d5-487c-aef0-24503bc840b2 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:52.279462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1424/2855] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:52 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:52.283882 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a3ae1-ecfa-428e-b680-7da4e644e464 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:52.294519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a3ae1-ecfa-428e-b680-7da4e644e464 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:52.349414 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298784b-de8b-4c83-b443-7b9904c1318c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:52.350019 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298784b-de8b-4c83-b443-7b9904c1318c admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:52.350296 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298784b-de8b-4c83-b443-7b9904c1318c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:52.350566 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298784b-de8b-4c83-b443-7b9904c1318c admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:52.351189 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298784b-de8b-4c83-b443-7b9904c1318c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:52.353144 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298784b-de8b-4c83-b443-7b9904c1318c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:52.376148 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1432/2856] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:52 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:52.382485 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac62e5-f4d5-487c-aef0-24503bc840b2 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:52.393624 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac62e5-f4d5-487c-aef0-24503bc840b2 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:52.398478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df470c3-21ed-4a84-b89a-1523f78df275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:52.399165 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df470c3-21ed-4a84-b89a-1523f78df275 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:52.399451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df470c3-21ed-4a84-b89a-1523f78df275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:52.399729 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df470c3-21ed-4a84-b89a-1523f78df275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:52.400410 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df470c3-21ed-4a84-b89a-1523f78df275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:52.402230 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df470c3-21ed-4a84-b89a-1523f78df275 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:52.413421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1425/2857] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:52.417721 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298784b-de8b-4c83-b443-7b9904c1318c admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:52.428561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298784b-de8b-4c83-b443-7b9904c1318c admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:52.433454 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db15a0-2b3f-41c2-9e90-a29e9604ddeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:52.434053 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db15a0-2b3f-41c2-9e90-a29e9604ddeb admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:52.434332 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db15a0-2b3f-41c2-9e90-a29e9604ddeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:52.435342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db15a0-2b3f-41c2-9e90-a29e9604ddeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:52.436024 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db15a0-2b3f-41c2-9e90-a29e9604ddeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:52.437629 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db15a0-2b3f-41c2-9e90-a29e9604ddeb admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:52.448592 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1433/2858] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:27:52.453369 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df470c3-21ed-4a84-b89a-1523f78df275 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:52.463451 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df470c3-21ed-4a84-b89a-1523f78df275 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:52.468376 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf93e20-012e-4d5b-a8ad-598c92b0328d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:52.469022 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf93e20-012e-4d5b-a8ad-598c92b0328d admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:52.469304 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf93e20-012e-4d5b-a8ad-598c92b0328d admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:52.469579 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf93e20-012e-4d5b-a8ad-598c92b0328d admin admin] PATH_INFO: `/v3/projects/c77077d3e8014e749feda2548801d66c/users/f4d4032b453f461e9f0989ea215acdf5/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:52.484600 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf93e20-012e-4d5b-a8ad-598c92b0328d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c77077d3e8014e749feda2548801d66c, user_id=f4d4032b453f461e9f0989ea215acdf5, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:52.487303 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf93e20-012e-4d5b-a8ad-598c92b0328d admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:52.501189 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1426/2859] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:52 2022] PUT /identity/v3/projects/c77077d3e8014e749feda2548801d66c/users/f4d4032b453f461e9f0989ea215acdf5/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:52.506178 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619488-56e3-42ab-8bd5-2e0e08a875a0 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:52.506462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619488-56e3-42ab-8bd5-2e0e08a875a0 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:52.506765 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619488-56e3-42ab-8bd5-2e0e08a875a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:52.526743 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-52619488-56e3-42ab-8bd5-2e0e08a875a0 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:52.527960 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-52619488-56e3-42ab-8bd5-2e0e08a875a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:52.552819 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:52.591776 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1434/2860] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:52.601470 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf93e20-012e-4d5b-a8ad-598c92b0328d admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:52.603151 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-4bf93e20-012e-4d5b-a8ad-598c92b0328d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:52.660423 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf93e20-012e-4d5b-a8ad-598c92b0328d admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:52.664012 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5d3b5-808a-4721-9a29-4427176d0633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:52.664621 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5d3b5-808a-4721-9a29-4427176d0633 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:52.664893 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5d3b5-808a-4721-9a29-4427176d0633 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:52.665164 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5d3b5-808a-4721-9a29-4427176d0633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:52.674041 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5d3b5-808a-4721-9a29-4427176d0633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:52.675449 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5d3b5-808a-4721-9a29-4427176d0633 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:52.689937 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1427/2861] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:53.328172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c71b10-8ae5-46ba-82ee-2721e176a6c4 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:53.328172 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c71b10-8ae5-46ba-82ee-2721e176a6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:53.328764 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c71b10-8ae5-46ba-82ee-2721e176a6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:53.359030 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-b3c71b10-8ae5-46ba-82ee-2721e176a6c4 None None] MFA Rules not processed for user `f4d4032b453f461e9f0989ea215acdf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:53.360405 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-b3c71b10-8ae5-46ba-82ee-2721e176a6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:53.434010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1435/2862] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:27:53.438919 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5d3b5-808a-4721-9a29-4427176d0633 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:53.448966 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5d3b5-808a-4721-9a29-4427176d0633 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:53.454481 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a51688-c766-48f7-98d4-6d7cf007a60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:53.454972 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51688-c766-48f7-98d4-6d7cf007a60f admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:53.455309 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51688-c766-48f7-98d4-6d7cf007a60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:53.455641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51688-c766-48f7-98d4-6d7cf007a60f admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:53.456385 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a51688-c766-48f7-98d4-6d7cf007a60f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:53.458265 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a51688-c766-48f7-98d4-6d7cf007a60f admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:53.491747 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1428/2863] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:27:53 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:27:53.493164 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c71b10-8ae5-46ba-82ee-2721e176a6c4 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:53.505243 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c71b10-8ae5-46ba-82ee-2721e176a6c4 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:53.565720 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e6b42-03f8-4707-ab3b-2eb5eb90d180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:53.566328 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e6b42-03f8-4707-ab3b-2eb5eb90d180 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:53.566601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e6b42-03f8-4707-ab3b-2eb5eb90d180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:53.566866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e6b42-03f8-4707-ab3b-2eb5eb90d180 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:53.567488 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e6b42-03f8-4707-ab3b-2eb5eb90d180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:53.569480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e6b42-03f8-4707-ab3b-2eb5eb90d180 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:53.591780 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1436/2864] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:27:53 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:27:53.596838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a51688-c766-48f7-98d4-6d7cf007a60f admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:53.607217 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a51688-c766-48f7-98d4-6d7cf007a60f admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:53.612109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bd31a-c1e2-4961-8f57-75d3c76f7766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:53.612707 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bd31a-c1e2-4961-8f57-75d3c76f7766 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:53.612992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bd31a-c1e2-4961-8f57-75d3c76f7766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:53.613263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bd31a-c1e2-4961-8f57-75d3c76f7766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:53.613938 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4bd31a-c1e2-4961-8f57-75d3c76f7766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:53.615511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4bd31a-c1e2-4961-8f57-75d3c76f7766 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:53.626695 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1429/2865] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:53.630885 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e6b42-03f8-4707-ab3b-2eb5eb90d180 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:53.641346 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e6b42-03f8-4707-ab3b-2eb5eb90d180 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:53.645896 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43efe9e-8809-4512-9c9d-55c842a331ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:53.646537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43efe9e-8809-4512-9c9d-55c842a331ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:53.646810 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43efe9e-8809-4512-9c9d-55c842a331ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:53.647073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43efe9e-8809-4512-9c9d-55c842a331ab admin admin] PATH_INFO: `/v3/projects/19b2eb8126fb4c55bfdcfa7f801123a1/users/95d65302e6014b5498ac62881ca5872b/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:53.663443 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43efe9e-8809-4512-9c9d-55c842a331ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b2eb8126fb4c55bfdcfa7f801123a1, user_id=95d65302e6014b5498ac62881ca5872b, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:53.666196 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43efe9e-8809-4512-9c9d-55c842a331ab admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:53.679291 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1437/2866] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:53 2022] PUT /identity/v3/projects/19b2eb8126fb4c55bfdcfa7f801123a1/users/95d65302e6014b5498ac62881ca5872b/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:53.683551 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bd31a-c1e2-4961-8f57-75d3c76f7766 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:53.694754 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bd31a-c1e2-4961-8f57-75d3c76f7766 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:53.722495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:53.753788 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849f27e-7eae-4645-b658-fba62fcee40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:53.754396 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849f27e-7eae-4645-b658-fba62fcee40c admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:53.754676 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849f27e-7eae-4645-b658-fba62fcee40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:53.754948 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849f27e-7eae-4645-b658-fba62fcee40c admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:53.755586 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849f27e-7eae-4645-b658-fba62fcee40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:53.757201 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849f27e-7eae-4645-b658-fba62fcee40c admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:53.767336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1430/2867] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:27:53.771726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43efe9e-8809-4512-9c9d-55c842a331ab admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:53.783253 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43efe9e-8809-4512-9c9d-55c842a331ab admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:53.787906 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2e7a3-6167-4de8-8c07-da6ef89831c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:53.788588 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2e7a3-6167-4de8-8c07-da6ef89831c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:53.788860 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2e7a3-6167-4de8-8c07-da6ef89831c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:53.789153 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2e7a3-6167-4de8-8c07-da6ef89831c8 admin admin] PATH_INFO: `/v3/projects/19b2eb8126fb4c55bfdcfa7f801123a1/users/95d65302e6014b5498ac62881ca5872b/roles/1d3cf6d1581e45c8aff54a0afd66dd68` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:53.791042 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2e7a3-6167-4de8-8c07-da6ef89831c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b2eb8126fb4c55bfdcfa7f801123a1, user_id=95d65302e6014b5498ac62881ca5872b, role_id=1d3cf6d1581e45c8aff54a0afd66dd68)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:53.793699 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2e7a3-6167-4de8-8c07-da6ef89831c8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:53.804643 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1438/2868] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:27:53 2022] PUT /identity/v3/projects/19b2eb8126fb4c55bfdcfa7f801123a1/users/95d65302e6014b5498ac62881ca5872b/roles/1d3cf6d1581e45c8aff54a0afd66dd68 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:54.445052 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d10af-f5f0-4897-9f84-736c733284b6 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:54.445536 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d10af-f5f0-4897-9f84-736c733284b6 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:54.445657 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d10af-f5f0-4897-9f84-736c733284b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:54.475109 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-683d10af-f5f0-4897-9f84-736c733284b6 None None] MFA Rules not processed for user `95d65302e6014b5498ac62881ca5872b`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:27:54.476421 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-683d10af-f5f0-4897-9f84-736c733284b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:54.548643 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1431/2869] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:27:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:27:54.560601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2e7a3-6167-4de8-8c07-da6ef89831c8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:54.562637 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-68d2e7a3-6167-4de8-8c07-da6ef89831c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:54.579186 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:54.628857 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2e7a3-6167-4de8-8c07-da6ef89831c8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:54.632529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0b58f-4b5e-4169-8196-705a9948ef18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:54.633145 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0b58f-4b5e-4169-8196-705a9948ef18 service cinder] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:54.633425 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0b58f-4b5e-4169-8196-705a9948ef18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:54.633694 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0b58f-4b5e-4169-8196-705a9948ef18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:54.689762 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0b58f-4b5e-4169-8196-705a9948ef18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:54.691211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0b58f-4b5e-4169-8196-705a9948ef18 service cinder] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:54.709496 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1439/2870] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:57.128373 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d10af-f5f0-4897-9f84-736c733284b6 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:57.138269 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d10af-f5f0-4897-9f84-736c733284b6 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:57.141963 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4d267-d31e-479e-a5dd-f3c7a830e827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:57.142558 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4d267-d31e-479e-a5dd-f3c7a830e827 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:57.142835 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4d267-d31e-479e-a5dd-f3c7a830e827 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:57.143106 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4d267-d31e-479e-a5dd-f3c7a830e827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:57.152080 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4d267-d31e-479e-a5dd-f3c7a830e827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:57.153504 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4d267-d31e-479e-a5dd-f3c7a830e827 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:57.164417 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1432/2871] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:28:04.704941 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=43 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:28:07.160141 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=56 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:28:08.074181 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0b58f-4b5e-4169-8196-705a9948ef18 service cinder] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:08.075653 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0da0b58f-4b5e-4169-8196-705a9948ef18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:08.147309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0b58f-4b5e-4169-8196-705a9948ef18 service cinder] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:08.151855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:08.152526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:08.152826 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:08.153091 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] PATH_INFO: `/v3/users/aca1b07c09b746a1941aada2c359b3a8` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:08.154259 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca1b07c09b746a1941aada2c359b3a8)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:08.156197 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:08.205273 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca1b07c09b746a1941aada2c359b3a8'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:08.217644 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca1b07c09b746a1941aada2c359b3a8'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:08.225079 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca1b07c09b746a1941aada2c359b3a8'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:08.232673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca1b07c09b746a1941aada2c359b3a8'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:08.233269 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca1b07c09b746a1941aada2c359b3a8'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:08.239827 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1440/2872] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:28:08 2022] DELETE /identity/v3/users/aca1b07c09b746a1941aada2c359b3a8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:28:08.244381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90273eb8-0130-4b1d-959c-ae9b79b3d85e None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:08.244895 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90273eb8-0130-4b1d-959c-ae9b79b3d85e None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:08.244994 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90273eb8-0130-4b1d-959c-ae9b79b3d85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:08.264047 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-90273eb8-0130-4b1d-959c-ae9b79b3d85e None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:28:08.265248 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-90273eb8-0130-4b1d-959c-ae9b79b3d85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:08.323170 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1433/2873] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:28:08.333233 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:08.334654 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:08.392302 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea36e2c-db11-4c11-b8f5-fd74bc403a97 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:08.395946 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1c1a9-7163-41b8-92be-a010ae6426c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:08.396583 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1c1a9-7163-41b8-92be-a010ae6426c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:08.396856 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1c1a9-7163-41b8-92be-a010ae6426c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:08.397128 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1c1a9-7163-41b8-92be-a010ae6426c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:08.406103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea1c1a9-7163-41b8-92be-a010ae6426c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:08.408217 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea1c1a9-7163-41b8-92be-a010ae6426c6 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:08.418073 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1441/2874] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:28:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:28:08.701014 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90273eb8-0130-4b1d-959c-ae9b79b3d85e None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:08.702550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-90273eb8-0130-4b1d-959c-ae9b79b3d85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:08.715647 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90273eb8-0130-4b1d-959c-ae9b79b3d85e None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:08.726462 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:08.727658 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:08.727806 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:08.728108 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] PATH_INFO: `/v3/projects/212385b1cc584ef4ae751d3ad3781fdd` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:08.729284 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=212385b1cc584ef4ae751d3ad3781fdd)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:08.731184 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:08.781081 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '212385b1cc584ef4ae751d3ad3781fdd', 'request_id': 'req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:08.788343 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '212385b1cc584ef4ae751d3ad3781fdd', 'request_id': 'req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:08.793787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '212385b1cc584ef4ae751d3ad3781fdd', 'request_id': 'req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:08.806718 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] The token cache is being invalidate because project 212385b1cc584ef4ae751d3ad3781fdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:28:08.807005 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:08.807627 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:08.809145 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1434/2875] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:28:08 2022] DELETE /identity/v3/projects/212385b1cc584ef4ae751d3ad3781fdd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:28:14.430519 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1c1a9-7163-41b8-92be-a010ae6426c6 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:14.432010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-cea1c1a9-7163-41b8-92be-a010ae6426c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:14.446735 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1c1a9-7163-41b8-92be-a010ae6426c6 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:14.465955 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:28:14.506422 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:14.507049 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:14.507325 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:14.507594 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] PATH_INFO: `/v3/users/f4d4032b453f461e9f0989ea215acdf5` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:14.508760 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4d4032b453f461e9f0989ea215acdf5)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:14.510724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:14.563913 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4d4032b453f461e9f0989ea215acdf5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:14.575903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4d4032b453f461e9f0989ea215acdf5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:14.583363 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4d4032b453f461e9f0989ea215acdf5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:14.590977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4d4032b453f461e9f0989ea215acdf5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:14.591663 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4d4032b453f461e9f0989ea215acdf5'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:14.598439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1442/2876] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:28:14 2022] DELETE /identity/v3/users/f4d4032b453f461e9f0989ea215acdf5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:28:14.775884 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:14.777438 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:14.792156 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76bdc0-ea69-4dee-9e01-25f9a52c20ed admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:14.851325 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:14.851956 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:14.852263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:14.852531 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] PATH_INFO: `/v3/projects/c77077d3e8014e749feda2548801d66c` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:14.853634 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c77077d3e8014e749feda2548801d66c)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:14.855537 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:14.887668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:28:14.905727 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c77077d3e8014e749feda2548801d66c', 'request_id': 'req-6f86a059-ac08-4ebc-be62-d312844a8f49'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:14.911874 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c77077d3e8014e749feda2548801d66c', 'request_id': 'req-6f86a059-ac08-4ebc-be62-d312844a8f49'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:14.917561 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c77077d3e8014e749feda2548801d66c', 'request_id': 'req-6f86a059-ac08-4ebc-be62-d312844a8f49'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:14.930103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] The token cache is being invalidate because project c77077d3e8014e749feda2548801d66c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:28:14.930397 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:14.931031 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:14.932559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1435/2877] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:28:14 2022] DELETE /identity/v3/projects/c77077d3e8014e749feda2548801d66c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:28:14.936668 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:14.938085 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:14.951724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16b2bb-ac76-42f3-8b05-fe92a0f5185f admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:15.008365 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4adbfe-e851-4f2a-b6a8-562d6aea9c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:15.009031 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4adbfe-e851-4f2a-b6a8-562d6aea9c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:15.009306 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4adbfe-e851-4f2a-b6a8-562d6aea9c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:15.009580 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4adbfe-e851-4f2a-b6a8-562d6aea9c86 admin admin] PATH_INFO: `/v3/users/95d65302e6014b5498ac62881ca5872b` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:15.010767 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4adbfe-e851-4f2a-b6a8-562d6aea9c86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d65302e6014b5498ac62881ca5872b)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:15.012737 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4adbfe-e851-4f2a-b6a8-562d6aea9c86 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:15.055266 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:28:15.058103 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-bd4adbfe-e851-4f2a-b6a8-562d6aea9c86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d65302e6014b5498ac62881ca5872b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:15.072520 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-bd4adbfe-e851-4f2a-b6a8-562d6aea9c86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d65302e6014b5498ac62881ca5872b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:15.079361 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-bd4adbfe-e851-4f2a-b6a8-562d6aea9c86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d65302e6014b5498ac62881ca5872b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:15.087616 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-bd4adbfe-e851-4f2a-b6a8-562d6aea9c86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d65302e6014b5498ac62881ca5872b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:15.088633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-bd4adbfe-e851-4f2a-b6a8-562d6aea9c86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d65302e6014b5498ac62881ca5872b'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:15.094689 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1443/2878] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:28:14 2022] DELETE /identity/v3/users/95d65302e6014b5498ac62881ca5872b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:28:15.272883 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:15.274326 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:15.288088 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86a059-ac08-4ebc-be62-d312844a8f49 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:15.345914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4fb6e-2531-4f05-8e03-47e53b49aef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:15.346542 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4fb6e-2531-4f05-8e03-47e53b49aef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:15.346820 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4fb6e-2531-4f05-8e03-47e53b49aef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:15.347092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4fb6e-2531-4f05-8e03-47e53b49aef5 admin admin] PATH_INFO: `/v3/projects/19b2eb8126fb4c55bfdcfa7f801123a1` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:15.348221 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4fb6e-2531-4f05-8e03-47e53b49aef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b2eb8126fb4c55bfdcfa7f801123a1)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:15.350127 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4fb6e-2531-4f05-8e03-47e53b49aef5 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:15.396428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-02e4fb6e-2531-4f05-8e03-47e53b49aef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b2eb8126fb4c55bfdcfa7f801123a1', 'request_id': 'req-02e4fb6e-2531-4f05-8e03-47e53b49aef5'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:15.402403 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-02e4fb6e-2531-4f05-8e03-47e53b49aef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b2eb8126fb4c55bfdcfa7f801123a1', 'request_id': 'req-02e4fb6e-2531-4f05-8e03-47e53b49aef5'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:15.407882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-02e4fb6e-2531-4f05-8e03-47e53b49aef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b2eb8126fb4c55bfdcfa7f801123a1', 'request_id': 'req-02e4fb6e-2531-4f05-8e03-47e53b49aef5'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:15.420475 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-02e4fb6e-2531-4f05-8e03-47e53b49aef5 admin admin] The token cache is being invalidate because project 19b2eb8126fb4c55bfdcfa7f801123a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:28:15.420791 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-02e4fb6e-2531-4f05-8e03-47e53b49aef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:15.421428 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-02e4fb6e-2531-4f05-8e03-47e53b49aef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:28:15.422918 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1436/2879] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:28:15 2022] DELETE /identity/v3/projects/19b2eb8126fb4c55bfdcfa7f801123a1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:28:15.428479 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87882d86-b65e-47c8-a6cf-8006e4eb5209 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:15.428759 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87882d86-b65e-47c8-a6cf-8006e4eb5209 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:15.429029 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87882d86-b65e-47c8-a6cf-8006e4eb5209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:15.448702 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-87882d86-b65e-47c8-a6cf-8006e4eb5209 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:28:15.449816 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-87882d86-b65e-47c8-a6cf-8006e4eb5209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:15.510303 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1444/2880] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:28:15.574777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f09b09-aac6-4f20-85de-8b53b8a99407 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:15.575064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f09b09-aac6-4f20-85de-8b53b8a99407 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:15.575339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f09b09-aac6-4f20-85de-8b53b8a99407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:15.595840 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-35f09b09-aac6-4f20-85de-8b53b8a99407 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:28:15.597104 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-35f09b09-aac6-4f20-85de-8b53b8a99407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:15.605167 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1437/2881] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:28:15.609543 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87882d86-b65e-47c8-a6cf-8006e4eb5209 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:15.619548 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87882d86-b65e-47c8-a6cf-8006e4eb5209 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:15.624097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d8212-5a9b-4273-81e1-fab8d2e716fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:15.624673 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d8212-5a9b-4273-81e1-fab8d2e716fa admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:15.624944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d8212-5a9b-4273-81e1-fab8d2e716fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:15.625208 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d8212-5a9b-4273-81e1-fab8d2e716fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:15.625849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d8212-5a9b-4273-81e1-fab8d2e716fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:28:15.626146 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d8212-5a9b-4273-81e1-fab8d2e716fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:15.628293 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d8212-5a9b-4273-81e1-fab8d2e716fa admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:15.640901 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1445/2882] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:28:15.646183 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f09b09-aac6-4f20-85de-8b53b8a99407 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:15.655437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f09b09-aac6-4f20-85de-8b53b8a99407 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:15.660211 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bcdc2-ec3a-4788-914f-d9d92eb46a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:15.660813 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bcdc2-ec3a-4788-914f-d9d92eb46a53 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:15.661092 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bcdc2-ec3a-4788-914f-d9d92eb46a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:15.661366 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bcdc2-ec3a-4788-914f-d9d92eb46a53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:15.661999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bcdc2-ec3a-4788-914f-d9d92eb46a53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:15.663920 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bcdc2-ec3a-4788-914f-d9d92eb46a53 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:15.691363 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1438/2883] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:28:15 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:28:15.695723 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d8212-5a9b-4273-81e1-fab8d2e716fa admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:15.706695 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d8212-5a9b-4273-81e1-fab8d2e716fa admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:15.764756 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23304f1a-bde2-47d1-8c6a-d996f598816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:15.765364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23304f1a-bde2-47d1-8c6a-d996f598816f admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:15.765640 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23304f1a-bde2-47d1-8c6a-d996f598816f admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:15.765905 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23304f1a-bde2-47d1-8c6a-d996f598816f admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:15.766537 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23304f1a-bde2-47d1-8c6a-d996f598816f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:15.769108 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23304f1a-bde2-47d1-8c6a-d996f598816f admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:15.790322 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1446/2884] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:28:15 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:28:15.794550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bcdc2-ec3a-4788-914f-d9d92eb46a53 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:15.804999 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bcdc2-ec3a-4788-914f-d9d92eb46a53 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:15.809686 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f9553-713a-41d7-aaa6-01173fddeacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:15.810345 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f9553-713a-41d7-aaa6-01173fddeacd admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:15.810611 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f9553-713a-41d7-aaa6-01173fddeacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:15.810914 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f9553-713a-41d7-aaa6-01173fddeacd admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:15.811617 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f9553-713a-41d7-aaa6-01173fddeacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:15.813413 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f9553-713a-41d7-aaa6-01173fddeacd admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:15.824116 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1439/2885] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:28:15.828724 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23304f1a-bde2-47d1-8c6a-d996f598816f admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:15.839518 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23304f1a-bde2-47d1-8c6a-d996f598816f admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:15.844096 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64d2e1-8efc-41d2-9b6a-84fc31e59d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:15.844687 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64d2e1-8efc-41d2-9b6a-84fc31e59d7c admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:15.844959 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64d2e1-8efc-41d2-9b6a-84fc31e59d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:15.845224 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64d2e1-8efc-41d2-9b6a-84fc31e59d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:15.846102 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64d2e1-8efc-41d2-9b6a-84fc31e59d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:15.847866 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64d2e1-8efc-41d2-9b6a-84fc31e59d7c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:15.858206 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1447/2886] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:28:15.862469 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f9553-713a-41d7-aaa6-01173fddeacd admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:15.868909 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:28:15.873060 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f9553-713a-41d7-aaa6-01173fddeacd admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:15.879591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695e43d-dafd-4056-b857-c38bc6b782c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:15.880547 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695e43d-dafd-4056-b857-c38bc6b782c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:15.881053 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695e43d-dafd-4056-b857-c38bc6b782c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:15.881591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695e43d-dafd-4056-b857-c38bc6b782c9 admin admin] PATH_INFO: `/v3/projects/6daea8caca73411abef8e14de09c5c54/users/282e92776c594100b697a79d0209be00/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:15.899833 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695e43d-dafd-4056-b857-c38bc6b782c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6daea8caca73411abef8e14de09c5c54, user_id=282e92776c594100b697a79d0209be00, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:15.902495 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695e43d-dafd-4056-b857-c38bc6b782c9 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:15.916467 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1440/2887] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:28:15 2022] PUT /identity/v3/projects/6daea8caca73411abef8e14de09c5c54/users/282e92776c594100b697a79d0209be00/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:28:15.922107 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470dae9-aef0-4be4-a94e-d349f021eb87 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:15.922462 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470dae9-aef0-4be4-a94e-d349f021eb87 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:15.922802 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470dae9-aef0-4be4-a94e-d349f021eb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:15.943741 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-7470dae9-aef0-4be4-a94e-d349f021eb87 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:28:15.945285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-7470dae9-aef0-4be4-a94e-d349f021eb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:15.966400 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:28:16.007624 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1448/2888] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:28:16.017401 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695e43d-dafd-4056-b857-c38bc6b782c9 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:16.019402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c695e43d-dafd-4056-b857-c38bc6b782c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:16.079694 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695e43d-dafd-4056-b857-c38bc6b782c9 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:16.083703 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210289d-b23d-4da5-867e-602e7908b795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:16.084406 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210289d-b23d-4da5-867e-602e7908b795 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:16.084662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210289d-b23d-4da5-867e-602e7908b795 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:16.084910 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210289d-b23d-4da5-867e-602e7908b795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:16.094254 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210289d-b23d-4da5-867e-602e7908b795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:16.095703 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210289d-b23d-4da5-867e-602e7908b795 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:16.105337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1441/2889] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:28:16.744273 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ce51-a4cb-4970-afbe-58c51fe2f801 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:16.744849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ce51-a4cb-4970-afbe-58c51fe2f801 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:16.744849 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ce51-a4cb-4970-afbe-58c51fe2f801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:16.777327 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-3453ce51-a4cb-4970-afbe-58c51fe2f801 None None] MFA Rules not processed for user `282e92776c594100b697a79d0209be00`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:28:16.778595 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3453ce51-a4cb-4970-afbe-58c51fe2f801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:16.853277 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1449/2890] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:28:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:28:16.864668 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210289d-b23d-4da5-867e-602e7908b795 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:16.866076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-b210289d-b23d-4da5-867e-602e7908b795 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:16.926735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210289d-b23d-4da5-867e-602e7908b795 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:16.930622 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7beef-2f55-428a-a0ac-c503b2e90e99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '534da5b47dda43dabaab2cb6aa683342', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:16.931230 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7beef-2f55-428a-a0ac-c503b2e90e99 service cinder] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:16.931507 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7beef-2f55-428a-a0ac-c503b2e90e99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:16.931924 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7beef-2f55-428a-a0ac-c503b2e90e99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:16.941559 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7beef-2f55-428a-a0ac-c503b2e90e99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:16.943017 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7beef-2f55-428a-a0ac-c503b2e90e99 service cinder] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:16.952111 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1442/2891] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:28:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:28:18.309852 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd5108-c125-4b1b-8dc2-c982866e1e5c None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:18.310331 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd5108-c125-4b1b-8dc2-c982866e1e5c None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:18.310444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd5108-c125-4b1b-8dc2-c982866e1e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:18.328894 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-f9cd5108-c125-4b1b-8dc2-c982866e1e5c None None] MFA Rules not processed for user `282e92776c594100b697a79d0209be00`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:28:18.330083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f9cd5108-c125-4b1b-8dc2-c982866e1e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:18.337450 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1450/2892] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:28:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:28:18.584603 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7beef-2f55-428a-a0ac-c503b2e90e99 service cinder] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:18.594024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7beef-2f55-428a-a0ac-c503b2e90e99 service cinder] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:18.597613 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db66b5-d40a-43a8-a91b-45cda98c128c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:18.598199 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db66b5-d40a-43a8-a91b-45cda98c128c service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:18.598477 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db66b5-d40a-43a8-a91b-45cda98c128c service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:18.598763 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db66b5-d40a-43a8-a91b-45cda98c128c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:18.600488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-58db66b5-d40a-43a8-a91b-45cda98c128c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:18.610437 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db66b5-d40a-43a8-a91b-45cda98c128c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:18.611887 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db66b5-d40a-43a8-a91b-45cda98c128c service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:18.619964 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1443/2893] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:28:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:28:18.962082 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd5108-c125-4b1b-8dc2-c982866e1e5c None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:18.963619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-f9cd5108-c125-4b1b-8dc2-c982866e1e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:19.048122 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd5108-c125-4b1b-8dc2-c982866e1e5c None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:19.053138 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecdce6-78da-4463-87ac-eafaf171a8f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08f34295f19b4d6ea405f1e8c8190db5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:19.053745 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecdce6-78da-4463-87ac-eafaf171a8f5 service placement] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:19.054022 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecdce6-78da-4463-87ac-eafaf171a8f5 service placement] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:19.054288 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecdce6-78da-4463-87ac-eafaf171a8f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:19.056018 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d2ecdce6-78da-4463-87ac-eafaf171a8f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:19.068492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecdce6-78da-4463-87ac-eafaf171a8f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:19.071100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecdce6-78da-4463-87ac-eafaf171a8f5 service placement] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:19.083522 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1451/2894] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:28:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:28:27.024050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db66b5-d40a-43a8-a91b-45cda98c128c service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:27.025834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-58db66b5-d40a-43a8-a91b-45cda98c128c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:27.104885 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db66b5-d40a-43a8-a91b-45cda98c128c service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:27.110587 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760784c-69a9-424c-8c25-4295d6b4afd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '049399a752ed4646a13f8e4a3a62d8e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:27.111442 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760784c-69a9-424c-8c25-4295d6b4afd5 service nova] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:27.111572 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760784c-69a9-424c-8c25-4295d6b4afd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:27.111822 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760784c-69a9-424c-8c25-4295d6b4afd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:27.114856 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0760784c-69a9-424c-8c25-4295d6b4afd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:27.127930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760784c-69a9-424c-8c25-4295d6b4afd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:27.129828 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760784c-69a9-424c-8c25-4295d6b4afd5 service nova] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:27.135348 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:28:27.143103 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1444/2895] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:28:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:28:29.077773 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=82 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:28:33.688289 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b783f46-2059-4e98-811e-48fe2c0b0e16 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:33.688907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b783f46-2059-4e98-811e-48fe2c0b0e16 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:33.688907 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b783f46-2059-4e98-811e-48fe2c0b0e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:33.709175 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-2b783f46-2059-4e98-811e-48fe2c0b0e16 None None] MFA Rules not processed for user `282e92776c594100b697a79d0209be00`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:28:33.710806 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-2b783f46-2059-4e98-811e-48fe2c0b0e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:33.717272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1452/2896] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:28:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:28:33.953550 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760784c-69a9-424c-8c25-4295d6b4afd5 service nova] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:33.955054 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-0760784c-69a9-424c-8c25-4295d6b4afd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:33.966960 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760784c-69a9-424c-8c25-4295d6b4afd5 service nova] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:33.970663 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811068d-e793-4cc6-922a-5bdfcb36b5c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:33.971253 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811068d-e793-4cc6-922a-5bdfcb36b5c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:33.971544 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811068d-e793-4cc6-922a-5bdfcb36b5c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:33.971787 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811068d-e793-4cc6-922a-5bdfcb36b5c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:33.973760 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c811068d-e793-4cc6-922a-5bdfcb36b5c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:33.983282 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811068d-e793-4cc6-922a-5bdfcb36b5c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:33.984702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811068d-e793-4cc6-922a-5bdfcb36b5c3 service neutron] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:33.993629 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1445/2897] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:28:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:28:43.702847 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:28:43.989215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=7 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:28:52.367002 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e0a40-7f0d-40d1-ab66-38b19262bbf2 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:52.367485 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e0a40-7f0d-40d1-ab66-38b19262bbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:52.367595 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e0a40-7f0d-40d1-ab66-38b19262bbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:52.419285 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-c62e0a40-7f0d-40d1-ab66-38b19262bbf2 None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:28:52.420531 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-c62e0a40-7f0d-40d1-ab66-38b19262bbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:52.500398 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1453/2898] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Aug 30 01:28:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:28:52.548746 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811068d-e793-4cc6-922a-5bdfcb36b5c3 service neutron] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:28:52.550237 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-c811068d-e793-4cc6-922a-5bdfcb36b5c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:28:52.641998 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811068d-e793-4cc6-922a-5bdfcb36b5c3 service neutron] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:28:52.647071 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38467852-97a3-4575-8b24-4c895a659ee1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:28:52.647671 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38467852-97a3-4575-8b24-4c895a659ee1 service swift] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:28:52.647953 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38467852-97a3-4575-8b24-4c895a659ee1 service swift] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:28:52.648259 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38467852-97a3-4575-8b24-4c895a659ee1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:28:52.657098 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38467852-97a3-4575-8b24-4c895a659ee1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:28:52.658788 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38467852-97a3-4575-8b24-4c895a659ee1 service swift] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:28:52.667332 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1446/2899] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:28:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:29:02.495898 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=40 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:29:02.663675 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=41 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:29:07.292711 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74938f0d-854e-4b26-bdec-26b86c2edc7b None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:07.293223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74938f0d-854e-4b26-bdec-26b86c2edc7b None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:07.293344 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74938f0d-854e-4b26-bdec-26b86c2edc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:07.313517 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-74938f0d-854e-4b26-bdec-26b86c2edc7b None None] MFA Rules not processed for user `8a3c3027b61d4f75a67e0a1bb8b52903`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:29:07.314753 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-74938f0d-854e-4b26-bdec-26b86c2edc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:07.322615 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1454/2900] 10.222.0.44 () {58 vars in 967 bytes} [Tue Aug 30 01:29:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:29:07.335023 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38467852-97a3-4575-8b24-4c895a659ee1 service swift] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:07.346064 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38467852-97a3-4575-8b24-4c895a659ee1 service swift] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:07.351083 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89ee0f-ff16-4569-921c-4dcb9a3c42fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3330df65b64c4fffbc3bb687d5c51021', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:07.351669 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89ee0f-ff16-4569-921c-4dcb9a3c42fb service swift] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:07.351935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89ee0f-ff16-4569-921c-4dcb9a3c42fb service swift] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:07.352246 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89ee0f-ff16-4569-921c-4dcb9a3c42fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:07.360702 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89ee0f-ff16-4569-921c-4dcb9a3c42fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:07.362252 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89ee0f-ff16-4569-921c-4dcb9a3c42fb service swift] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:07.370710 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1447/2901] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Aug 30 01:29:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:29:09.294778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74938f0d-854e-4b26-bdec-26b86c2edc7b None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:09.307261 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74938f0d-854e-4b26-bdec-26b86c2edc7b None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:09.311794 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f615455-6965-415e-8a95-ec508fc63c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:09.312444 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f615455-6965-415e-8a95-ec508fc63c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:09.312716 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f615455-6965-415e-8a95-ec508fc63c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:09.312980 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f615455-6965-415e-8a95-ec508fc63c53 admin admin] PATH_INFO: `/v3/users/282e92776c594100b697a79d0209be00` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:09.314077 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f615455-6965-415e-8a95-ec508fc63c53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=282e92776c594100b697a79d0209be00)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:09.315979 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f615455-6965-415e-8a95-ec508fc63c53 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:09.369276 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3f615455-6965-415e-8a95-ec508fc63c53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '282e92776c594100b697a79d0209be00'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:09.382222 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3f615455-6965-415e-8a95-ec508fc63c53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '282e92776c594100b697a79d0209be00'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:09.390058 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3f615455-6965-415e-8a95-ec508fc63c53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '282e92776c594100b697a79d0209be00'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:09.397738 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3f615455-6965-415e-8a95-ec508fc63c53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '282e92776c594100b697a79d0209be00'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:09.398526 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3f615455-6965-415e-8a95-ec508fc63c53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '282e92776c594100b697a79d0209be00'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:09.405360 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1455/2902] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:29:09 2022] DELETE /identity/v3/users/282e92776c594100b697a79d0209be00 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:29:09.581232 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89ee0f-ff16-4569-921c-4dcb9a3c42fb service swift] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:09.594338 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89ee0f-ff16-4569-921c-4dcb9a3c42fb service swift] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:09.654172 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:09.654823 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:09.655157 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:09.655487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58 admin admin] PATH_INFO: `/v3/projects/6daea8caca73411abef8e14de09c5c54` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:09.656701 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6daea8caca73411abef8e14de09c5c54)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:09.658577 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:09.705350 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6daea8caca73411abef8e14de09c5c54', 'request_id': 'req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:09.711738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6daea8caca73411abef8e14de09c5c54', 'request_id': 'req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:09.718112 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6daea8caca73411abef8e14de09c5c54', 'request_id': 'req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:09.730928 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58 admin admin] The token cache is being invalidate because project 6daea8caca73411abef8e14de09c5c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:29:09.731215 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:09.731834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-012aa3f3-8bf4-47f0-bc77-6b1611c04a58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:09.733337 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1448/2903] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:29:09 2022] DELETE /identity/v3/projects/6daea8caca73411abef8e14de09c5c54 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:29:09.738324 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee74f05-ef20-4366-a792-9343353d7cdc None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:09.738604 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee74f05-ef20-4366-a792-9343353d7cdc None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:09.738872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee74f05-ef20-4366-a792-9343353d7cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:09.759086 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-aee74f05-ef20-4366-a792-9343353d7cdc None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:29:09.761003 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-aee74f05-ef20-4366-a792-9343353d7cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:09.820665 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1456/2904] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:29:09.829378 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f3754-f8b5-47a0-9492-65add6c77da2 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:09.829662 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f3754-f8b5-47a0-9492-65add6c77da2 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:09.829935 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f3754-f8b5-47a0-9492-65add6c77da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:09.850257 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-089f3754-f8b5-47a0-9492-65add6c77da2 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:29:09.851494 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-089f3754-f8b5-47a0-9492-65add6c77da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:09.860402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1449/2905] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:29:09.864758 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee74f05-ef20-4366-a792-9343353d7cdc None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:09.876646 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee74f05-ef20-4366-a792-9343353d7cdc None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:09.883529 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86258f59-5633-435e-ba6e-7129e03ec3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:09.884199 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258f59-5633-435e-ba6e-7129e03ec3bf admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:09.884479 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258f59-5633-435e-ba6e-7129e03ec3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:09.884751 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258f59-5633-435e-ba6e-7129e03ec3bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:09.885566 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86258f59-5633-435e-ba6e-7129e03ec3bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:29:09.885855 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86258f59-5633-435e-ba6e-7129e03ec3bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:09.887817 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86258f59-5633-435e-ba6e-7129e03ec3bf admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:09.900682 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1457/2906] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:29:09.905641 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f3754-f8b5-47a0-9492-65add6c77da2 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:09.915124 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f3754-f8b5-47a0-9492-65add6c77da2 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:09.920472 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5326cd8-9b9d-4ade-ae6b-b85532c513b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:09.921066 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5326cd8-9b9d-4ade-ae6b-b85532c513b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:09.921339 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5326cd8-9b9d-4ade-ae6b-b85532c513b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:09.921604 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5326cd8-9b9d-4ade-ae6b-b85532c513b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:09.922241 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5326cd8-9b9d-4ade-ae6b-b85532c513b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:09.924146 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5326cd8-9b9d-4ade-ae6b-b85532c513b4 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:09.951409 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1450/2907] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:29:09 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:29:09.955619 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86258f59-5633-435e-ba6e-7129e03ec3bf admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:09.966423 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86258f59-5633-435e-ba6e-7129e03ec3bf admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:10.013925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:29:10.023838 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df05b5e-0796-4a0a-9b9c-70b2cbcdaff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:10.024479 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df05b5e-0796-4a0a-9b9c-70b2cbcdaff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:10.024757 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df05b5e-0796-4a0a-9b9c-70b2cbcdaff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:10.025027 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df05b5e-0796-4a0a-9b9c-70b2cbcdaff8 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:10.025666 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df05b5e-0796-4a0a-9b9c-70b2cbcdaff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:10.028412 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df05b5e-0796-4a0a-9b9c-70b2cbcdaff8 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:10.049505 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1458/2908] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:29:09 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:29:10.053740 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5326cd8-9b9d-4ade-ae6b-b85532c513b4 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:10.066456 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5326cd8-9b9d-4ade-ae6b-b85532c513b4 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:10.071287 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdecb27-bb86-43b0-92dd-2ee04c46437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:10.071811 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdecb27-bb86-43b0-92dd-2ee04c46437a admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:10.072163 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdecb27-bb86-43b0-92dd-2ee04c46437a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:10.072382 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdecb27-bb86-43b0-92dd-2ee04c46437a admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:10.074087 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdecb27-bb86-43b0-92dd-2ee04c46437a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:10.074908 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdecb27-bb86-43b0-92dd-2ee04c46437a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:10.085101 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1451/2909] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:29:10.089792 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df05b5e-0796-4a0a-9b9c-70b2cbcdaff8 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:10.099677 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df05b5e-0796-4a0a-9b9c-70b2cbcdaff8 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:10.105918 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea7a74-a229-41c2-95d9-ed4648a1642c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:10.106507 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea7a74-a229-41c2-95d9-ed4648a1642c admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:10.106778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea7a74-a229-41c2-95d9-ed4648a1642c admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:10.107042 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea7a74-a229-41c2-95d9-ed4648a1642c admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:10.107678 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea7a74-a229-41c2-95d9-ed4648a1642c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:10.109309 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea7a74-a229-41c2-95d9-ed4648a1642c admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:10.119830 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1459/2910] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:29:10.123929 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdecb27-bb86-43b0-92dd-2ee04c46437a admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:10.134487 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdecb27-bb86-43b0-92dd-2ee04c46437a admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:10.139802 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bae1f5-ba8c-43d7-a18c-4912a9e09853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:10.140489 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bae1f5-ba8c-43d7-a18c-4912a9e09853 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:10.140800 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bae1f5-ba8c-43d7-a18c-4912a9e09853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:10.141075 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bae1f5-ba8c-43d7-a18c-4912a9e09853 admin admin] PATH_INFO: `/v3/projects/ac84c3b2018e46ee8d355e7d105948fc/users/bb35ba0893a143ee8a15122316de02eb/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:10.156270 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bae1f5-ba8c-43d7-a18c-4912a9e09853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac84c3b2018e46ee8d355e7d105948fc, user_id=bb35ba0893a143ee8a15122316de02eb, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:10.158834 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bae1f5-ba8c-43d7-a18c-4912a9e09853 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:10.172076 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1452/2911] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:29:10 2022] PUT /identity/v3/projects/ac84c3b2018e46ee8d355e7d105948fc/users/bb35ba0893a143ee8a15122316de02eb/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:29:10.182873 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde51934-91e5-4627-830c-0e575fd91343 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:10.183185 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde51934-91e5-4627-830c-0e575fd91343 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:10.183393 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde51934-91e5-4627-830c-0e575fd91343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:10.211203 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-fde51934-91e5-4627-830c-0e575fd91343 None None] MFA Rules not processed for user `bb35ba0893a143ee8a15122316de02eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:29:10.212465 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-fde51934-91e5-4627-830c-0e575fd91343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:10.280064 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1460/2912] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:29:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:29:10.292329 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bae1f5-ba8c-43d7-a18c-4912a9e09853 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:10.294048 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-93bae1f5-ba8c-43d7-a18c-4912a9e09853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:10.312180 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:29:10.374125 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bae1f5-ba8c-43d7-a18c-4912a9e09853 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:10.378879 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a485d6-a092-423a-9c72-29ec334d6a4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c9d444b3294ed18c3d488cf0a3eeb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:10.379502 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a485d6-a092-423a-9c72-29ec334d6a4f service glance] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:10.379777 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a485d6-a092-423a-9c72-29ec334d6a4f service glance] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:10.380078 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a485d6-a092-423a-9c72-29ec334d6a4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:10.389679 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a485d6-a092-423a-9c72-29ec334d6a4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:10.391093 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a485d6-a092-423a-9c72-29ec334d6a4f service glance] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:10.399971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1453/2913] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:29:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:29:20.275903 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:29:20.395882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=29 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:29:40.805499 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde51934-91e5-4627-830c-0e575fd91343 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:40.867577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde51934-91e5-4627-830c-0e575fd91343 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:40.872118 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:40.872749 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:40.873028 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:40.873300 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] PATH_INFO: `/v3/users/bb35ba0893a143ee8a15122316de02eb` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:40.874490 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb35ba0893a143ee8a15122316de02eb)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:40.876466 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:40.924717 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb35ba0893a143ee8a15122316de02eb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:40.936605 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb35ba0893a143ee8a15122316de02eb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:40.943977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb35ba0893a143ee8a15122316de02eb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:40.951373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb35ba0893a143ee8a15122316de02eb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:40.952010 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb35ba0893a143ee8a15122316de02eb'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:40.958219 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1461/2914] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:29:40 2022] DELETE /identity/v3/users/bb35ba0893a143ee8a15122316de02eb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:29:40.962742 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620384d-7ad8-46c2-b3ba-1ce6e7365ef0 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:40.963298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620384d-7ad8-46c2-b3ba-1ce6e7365ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:40.963298 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620384d-7ad8-46c2-b3ba-1ce6e7365ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:40.982592 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-f620384d-7ad8-46c2-b3ba-1ce6e7365ef0 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:29:40.984522 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f620384d-7ad8-46c2-b3ba-1ce6e7365ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:41.044651 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1454/2915] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:29:41.054489 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:41.055884 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:41.114159 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc820c-46e0-4a23-828e-02411de01f8f admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:41.117814 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee6b45-9130-456a-8543-442f7cc70697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:41.118419 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee6b45-9130-456a-8543-442f7cc70697 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:41.118696 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee6b45-9130-456a-8543-442f7cc70697 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:41.118966 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee6b45-9130-456a-8543-442f7cc70697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:41.128569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee6b45-9130-456a-8543-442f7cc70697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:41.130007 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee6b45-9130-456a-8543-442f7cc70697 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:41.139319 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1462/2916] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:29:41.424716 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620384d-7ad8-46c2-b3ba-1ce6e7365ef0 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:41.426144 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-f620384d-7ad8-46c2-b3ba-1ce6e7365ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:41.437992 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620384d-7ad8-46c2-b3ba-1ce6e7365ef0 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:41.450985 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d66c79-9459-4b6b-8291-30ff9a16ca33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:41.451638 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d66c79-9459-4b6b-8291-30ff9a16ca33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:41.451916 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d66c79-9459-4b6b-8291-30ff9a16ca33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:41.452838 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d66c79-9459-4b6b-8291-30ff9a16ca33 admin admin] PATH_INFO: `/v3/projects/ac84c3b2018e46ee8d355e7d105948fc` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:41.453402 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d66c79-9459-4b6b-8291-30ff9a16ca33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac84c3b2018e46ee8d355e7d105948fc)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:41.455381 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d66c79-9459-4b6b-8291-30ff9a16ca33 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:41.505588 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d5d66c79-9459-4b6b-8291-30ff9a16ca33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac84c3b2018e46ee8d355e7d105948fc', 'request_id': 'req-d5d66c79-9459-4b6b-8291-30ff9a16ca33'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:41.511555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d5d66c79-9459-4b6b-8291-30ff9a16ca33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac84c3b2018e46ee8d355e7d105948fc', 'request_id': 'req-d5d66c79-9459-4b6b-8291-30ff9a16ca33'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:41.517419 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d5d66c79-9459-4b6b-8291-30ff9a16ca33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac84c3b2018e46ee8d355e7d105948fc', 'request_id': 'req-d5d66c79-9459-4b6b-8291-30ff9a16ca33'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:41.529978 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d5d66c79-9459-4b6b-8291-30ff9a16ca33 admin admin] The token cache is being invalidate because project ac84c3b2018e46ee8d355e7d105948fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:29:41.530292 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d5d66c79-9459-4b6b-8291-30ff9a16ca33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:41.531786 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-d5d66c79-9459-4b6b-8291-30ff9a16ca33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:29:41.533478 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1455/2917] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:29:41 2022] DELETE /identity/v3/projects/ac84c3b2018e46ee8d355e7d105948fc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:29:41.538387 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae41199-98bd-48b9-9ca2-11000baf9474 None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:41.538671 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae41199-98bd-48b9-9ca2-11000baf9474 None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:41.538944 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae41199-98bd-48b9-9ca2-11000baf9474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:41.560097 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-3ae41199-98bd-48b9-9ca2-11000baf9474 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:29:41.561292 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3ae41199-98bd-48b9-9ca2-11000baf9474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:41.578364 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:29:41.624726 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1463/2918] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:29:41.632620 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea079b5c-136f-412b-a383-fc131e262943 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:41.632902 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea079b5c-136f-412b-a383-fc131e262943 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:41.633176 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea079b5c-136f-412b-a383-fc131e262943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:41.652773 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-ea079b5c-136f-412b-a383-fc131e262943 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:29:41.653971 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-ea079b5c-136f-412b-a383-fc131e262943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:41.662738 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1456/2919] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:29:41.667167 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae41199-98bd-48b9-9ca2-11000baf9474 None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:41.677480 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae41199-98bd-48b9-9ca2-11000baf9474 None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:41.682013 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f897be7-2cb3-42d0-8c7e-faf0205a20fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:41.682601 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f897be7-2cb3-42d0-8c7e-faf0205a20fb admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:41.682872 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f897be7-2cb3-42d0-8c7e-faf0205a20fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:41.683136 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f897be7-2cb3-42d0-8c7e-faf0205a20fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:41.683775 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f897be7-2cb3-42d0-8c7e-faf0205a20fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81647) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:29:41.684106 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f897be7-2cb3-42d0-8c7e-faf0205a20fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:41.689225 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f897be7-2cb3-42d0-8c7e-faf0205a20fb admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:41.697272 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1464/2920] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Aug 30 01:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:29:41.701726 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea079b5c-136f-412b-a383-fc131e262943 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:41.711944 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea079b5c-136f-412b-a383-fc131e262943 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:41.716853 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24da000-7694-46e0-9a59-b0ac9d93779a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:41.717463 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24da000-7694-46e0-9a59-b0ac9d93779a admin admin] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:41.717739 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24da000-7694-46e0-9a59-b0ac9d93779a admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:41.718010 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24da000-7694-46e0-9a59-b0ac9d93779a admin admin] PATH_INFO: `/v3/projects` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:41.719166 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24da000-7694-46e0-9a59-b0ac9d93779a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:41.721196 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24da000-7694-46e0-9a59-b0ac9d93779a admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:41.750050 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1457/2921] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Aug 30 01:29:41 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:29:41.754582 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f897be7-2cb3-42d0-8c7e-faf0205a20fb admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:41.764710 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f897be7-2cb3-42d0-8c7e-faf0205a20fb admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:41.826840 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727e4cd-53eb-475f-9d88-93b7455494f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:41.828074 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727e4cd-53eb-475f-9d88-93b7455494f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:41.828820 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727e4cd-53eb-475f-9d88-93b7455494f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:41.829100 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727e4cd-53eb-475f-9d88-93b7455494f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:41.829742 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727e4cd-53eb-475f-9d88-93b7455494f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:41.831659 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727e4cd-53eb-475f-9d88-93b7455494f5 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:41.853492 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1465/2922] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Aug 30 01:29:41 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:29:41.857580 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24da000-7694-46e0-9a59-b0ac9d93779a admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:41.868555 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24da000-7694-46e0-9a59-b0ac9d93779a admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:41.873084 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d75773-7764-4c08-9bb9-ef04c10dc8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:41.873671 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d75773-7764-4c08-9bb9-ef04c10dc8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:41.873939 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d75773-7764-4c08-9bb9-ef04c10dc8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:41.874208 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d75773-7764-4c08-9bb9-ef04c10dc8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:41.875484 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d75773-7764-4c08-9bb9-ef04c10dc8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:41.877086 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d75773-7764-4c08-9bb9-ef04c10dc8f3 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:41.886842 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1458/2923] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:29:41.891083 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727e4cd-53eb-475f-9d88-93b7455494f5 admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:41.901484 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727e4cd-53eb-475f-9d88-93b7455494f5 admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:41.906778 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0455bd-6c0e-4f10-9d99-2e31c154f373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:41.907416 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0455bd-6c0e-4f10-9d99-2e31c154f373 admin admin] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:41.907662 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0455bd-6c0e-4f10-9d99-2e31c154f373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:41.908052 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0455bd-6c0e-4f10-9d99-2e31c154f373 admin admin] PATH_INFO: `/v3/roles` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:41.908693 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0455bd-6c0e-4f10-9d99-2e31c154f373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:41.911569 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0455bd-6c0e-4f10-9d99-2e31c154f373 admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:41.923633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1466/2924] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Aug 30 01:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:29:41.927799 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d75773-7764-4c08-9bb9-ef04c10dc8f3 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:41.938506 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d75773-7764-4c08-9bb9-ef04c10dc8f3 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:41.943089 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fbcc4-3bbb-4503-8be6-121f7bae25e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:41.943731 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fbcc4-3bbb-4503-8be6-121f7bae25e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:41.944015 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fbcc4-3bbb-4503-8be6-121f7bae25e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:41.944301 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fbcc4-3bbb-4503-8be6-121f7bae25e1 admin admin] PATH_INFO: `/v3/projects/559991bb4ca747e3b4884250e259f755/users/0fb514338ae440a384d691a59b884775/roles/b07ea738cc204b2b8125bf4cd6d9c54a` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:41.958770 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fbcc4-3bbb-4503-8be6-121f7bae25e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=559991bb4ca747e3b4884250e259f755, user_id=0fb514338ae440a384d691a59b884775, role_id=b07ea738cc204b2b8125bf4cd6d9c54a)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:41.961424 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fbcc4-3bbb-4503-8be6-121f7bae25e1 admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:41.974677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1459/2925] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Aug 30 01:29:41 2022] PUT /identity/v3/projects/559991bb4ca747e3b4884250e259f755/users/0fb514338ae440a384d691a59b884775/roles/b07ea738cc204b2b8125bf4cd6d9c54a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:29:41.986939 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4a59b-1d63-4700-b8e0-ea1e389ad13f None None] REQUEST_METHOD: `POST` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:41.987223 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4a59b-1d63-4700-b8e0-ea1e389ad13f None None] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:41.987498 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4a59b-1d63-4700-b8e0-ea1e389ad13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:42.015937 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.auth.core [None req-18d4a59b-1d63-4700-b8e0-ea1e389ad13f None None] MFA Rules not processed for user `0fb514338ae440a384d691a59b884775`. Rule list: `[]` (Enabled: `True`). {{(pid=81647) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:29:42.017373 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-18d4a59b-1d63-4700-b8e0-ea1e389ad13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:42.085578 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1467/2926] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:29:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:29:42.098016 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fbcc4-3bbb-4503-8be6-121f7bae25e1 admin admin] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:42.099930 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-2c2fbcc4-3bbb-4503-8be6-121f7bae25e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:42.122970 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:29:42.160869 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fbcc4-3bbb-4503-8be6-121f7bae25e1 admin admin] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:42.164525 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a01ddd-f830-471b-959c-4248a180e02b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c9d444b3294ed18c3d488cf0a3eeb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:42.165126 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a01ddd-f830-471b-959c-4248a180e02b service glance] REQUEST_METHOD: `GET` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:42.165405 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a01ddd-f830-471b-959c-4248a180e02b service glance] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:42.165677 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a01ddd-f830-471b-959c-4248a180e02b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:42.174420 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a01ddd-f830-471b-959c-4248a180e02b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:42.175847 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a01ddd-f830-471b-959c-4248a180e02b service glance] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:42.184622 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1460/2927] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:29:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:29:52.082565 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:29:52.180591 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=17 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:30:06.846754 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4a59b-1d63-4700-b8e0-ea1e389ad13f None None] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:30:06.909577 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4a59b-1d63-4700-b8e0-ea1e389ad13f None None] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:30:06.914220 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:30:06.914851 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:30:06.915129 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:30:06.915404 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] PATH_INFO: `/v3/users/0fb514338ae440a384d691a59b884775` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:30:06.916549 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb514338ae440a384d691a59b884775)` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:30:06.918465 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:30:06.965456 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb514338ae440a384d691a59b884775'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:30:06.977750 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb514338ae440a384d691a59b884775'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:30:06.984494 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb514338ae440a384d691a59b884775'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:30:06.992439 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb514338ae440a384d691a59b884775'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:30:06.993054 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.notifications [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb514338ae440a384d691a59b884775'} {{(pid=81647) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:30:06.999211 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1468/2928] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Aug 30 01:30:06 2022] DELETE /identity/v3/users/0fb514338ae440a384d691a59b884775 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:30:07.003860 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd4de8-d288-4e08-9c2c-515779599908 None None] REQUEST_METHOD: `POST` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:30:07.004399 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd4de8-d288-4e08-9c2c-515779599908 None None] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:30:07.004514 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd4de8-d288-4e08-9c2c-515779599908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:30:07.023801 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.auth.core [None req-9ccd4de8-d288-4e08-9c2c-515779599908 None None] MFA Rules not processed for user `56560cd65ed74a0a9e8aff03655fb0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81646) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:30:07.025024 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9ccd4de8-d288-4e08-9c2c-515779599908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:30:07.083818 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1461/2929] 10.222.0.44 () {58 vars in 958 bytes} [Tue Aug 30 01:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:30:07.093429 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:30:07.094833 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:30:07.152510 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39013a-d975-4d43-b813-16095749b2fd admin admin] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:30:07.156266 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1248a84-750a-4a81-87fc-8f7ffb63f9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cea2f9238b42e0a7cb26bac494a8c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:30:07.156799 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1248a84-750a-4a81-87fc-8f7ffb63f9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:30:07.157076 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1248a84-750a-4a81-87fc-8f7ffb63f9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:30:07.157342 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1248a84-750a-4a81-87fc-8f7ffb63f9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:30:07.167038 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1248a84-750a-4a81-87fc-8f7ffb63f9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:30:07.168502 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1248a84-750a-4a81-87fc-8f7ffb63f9d3 service neutron] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:30:07.177831 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1469/2930] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:30:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:30:07.456623 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd4de8-d288-4e08-9c2c-515779599908 None None] Authenticating user token {{(pid=81646) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:30:07.458063 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.fernet_utils [None req-9ccd4de8-d288-4e08-9c2c-515779599908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81646) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:30:07.470488 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd4de8-d288-4e08-9c2c-515779599908 None None] Validating token access rules against request {{(pid=81646) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:30:07.475263 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56560cd65ed74a0a9e8aff03655fb0e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '236d122dbb6b4283a260bcccbdcd4a4a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81646) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:30:07.475925 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:30:07.476236 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:30:07.476511 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b admin admin] PATH_INFO: `/v3/projects/559991bb4ca747e3b4884250e259f755` {{(pid=81646) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:30:07.477645 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b admin admin] RBAC: Authorizing `identity:delete_project(project_id=559991bb4ca747e3b4884250e259f755)` {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:30:07.479568 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b admin admin] RBAC: Authorization granted {{(pid=81646) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:30:07.528819 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '559991bb4ca747e3b4884250e259f755', 'request_id': 'req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:30:07.535619 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '559991bb4ca747e3b4884250e259f755', 'request_id': 'req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:30:07.541300 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '559991bb4ca747e3b4884250e259f755', 'request_id': 'req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b'} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:30:07.553882 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b admin admin] The token cache is being invalidate because project 559991bb4ca747e3b4884250e259f755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81646) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:30:07.554171 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:30:07.554810 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG keystone.notifications [None req-8cb8344f-26f7-43c7-b239-2d03fe5ec13b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81646) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:30:07.556336 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: [pid: 81646|app: 0|req: 1462/2931] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Aug 30 01:30:07 2022] DELETE /identity/v3/projects/559991bb4ca747e3b4884250e259f755 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:30:17.173977 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:30:17.550735 dv-cinder-815145-12-iscsi devstack@keystone.service[81646]: DEBUG dbcounter [-] [81646] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81646) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:30:32.743072 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1248a84-750a-4a81-87fc-8f7ffb63f9d3 service neutron] Authenticating user token {{(pid=81647) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:30:32.744854 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-d1248a84-750a-4a81-87fc-8f7ffb63f9d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:30:32.821561 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1248a84-750a-4a81-87fc-8f7ffb63f9d3 service neutron] Validating token access rules against request {{(pid=81647) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:30:32.886300 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a047c0-0c59-4e1b-a1c9-2886287a5318 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08f34295f19b4d6ea405f1e8c8190db5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9188023bf51d4737b801e3fbd9af95b1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81647) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:30:32.886925 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a047c0-0c59-4e1b-a1c9-2886287a5318 service placement] REQUEST_METHOD: `GET` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:30:32.887207 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a047c0-0c59-4e1b-a1c9-2886287a5318 service placement] SCRIPT_NAME: `/identity` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:30:32.887483 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a047c0-0c59-4e1b-a1c9-2886287a5318 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81647) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:30:32.889271 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.fernet_utils [None req-42a047c0-0c59-4e1b-a1c9-2886287a5318 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81647) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:30:32.901957 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a047c0-0c59-4e1b-a1c9-2886287a5318 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:30:32.903564 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a047c0-0c59-4e1b-a1c9-2886287a5318 service placement] RBAC: Authorization granted {{(pid=81647) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:30:32.916914 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: [pid: 81647|app: 0|req: 1470/2932] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Aug 30 01:30:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:30:42.910633 dv-cinder-815145-12-iscsi devstack@keystone.service[81647]: DEBUG dbcounter [-] [81647] Writing DB stats keystone:SELECT=58 {{(pid=81647) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}